Hi,
this has been subject to a couple of threads on python-dev already, for
example:
http://thread.gmane.org/gmane.comp.python.devel/135764/focus=140986http://thread.gmane.org/gmane.comp.python.devel/141037/focus=141046
It originally came out of issues 13429 and 16392.
http://bugs.python.org/issue13429http://bugs.python.org/issue16392
Here's an initial attempt at a PEP for it. It is based on the (unfinished)
ModuleSpec PEP, which is being discussed on the import-sig mailing list.
http://mail.python.org/pipermail/import-sig/2013-August/000688.html
Stefan
PEP: 4XX
Title: Redesigning extension modules
Version: $Revision$
Last-Modified: $Date$
Author: Stefan Behnel <stefan_ml at behnel.de>
BDFL-Delegate: ???
Discussions-To: ???
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 11-Aug-2013
Python-Version: 3.4
Post-History: 23-Aug-2013
Resolution:
Abstract
========
This PEP proposes a redesign of the way in which extension modules interact
with the interpreter runtime. This was last revised for Python 3.0 in PEP
3121, but did not solve all problems at the time. The goal is to solve them
by bringing extension modules closer to the way Python modules behave.
An implication of this PEP is that extension modules can use arbitrary
types for their module implementation and are no longer restricted to
types.ModuleType. This makes it easy to support properties at the module
level and to safely store arbitrary global state in the module that is
covered by normal garbage collection and supports reloading and
sub-interpreters.
Motivation
==========
Python modules and extension modules are not being set up in the same way.
For Python modules, the module is created and set up first, then the module
code is being executed. For extensions, i.e. shared libraries, the module
init function is executed straight away and does both the creation and
initialisation. This means that it knows neither the __file__ it is being
loaded from nor its package (i.e. its fully qualified module name, FQMN).
This hinders relative imports and resource loading. In Py3, it's also not
being added to sys.modules, which means that a (potentially transitive)
re-import of the module will really try to reimport it and thus run into an
infinite loop when it executes the module init function again. And without
the FQMN, it is not trivial to correctly add the module to sys.modules either.
This is specifically a problem for Cython generated modules, for which it's
not uncommon that the module init code has the same level of complexity as
that of any 'regular' Python module. Also, the lack of a FQMN and correct
file path hinders the compilation of __init__.py modules, i.e. packages,
especially when relative imports are being used at module init time.
Furthermore, the majority of currently existing extension modules has
problems with sub-interpreter support and/or reloading and it is neither
easy nor efficient with the current infrastructure to support these
features. This PEP also addresses these issues.
The current process
===================
Currently, extension modules export an initialisation function named
"PyInit_modulename", named after the file name of the shared library. This
function is executed by the import machinery and must return either NULL in
the case of an exception, or a fully initialised module object. The
function receives no arguments, so it has no way of knowing about its
import context.
During its execution, the module init function creates a module object
based on a PyModuleDef struct. It then continues to initialise it by adding
attributes to the module dict, creating types, etc.
In the back, the shared library loader keeps a note of the fully qualified
module name of the last module that it loaded, and when a module gets
created that has a matching name, this global variable is used to determine
the FQMN of the module object. This is not entirely safe as it relies on
the module init function creating its own module object first, but this
assumption usually holds in practice.
The main problem in this process is the missing support for passing state
into the module init function, and for safely passing state through to the
module creation code.
The proposal
============
The current extension module initialisation will be deprecated in favour of
a new initialisation scheme. Since the current scheme will continue to be
available, existing code will continue to work unchanged, including binary
compatibility.
Extension modules that support the new initialisation scheme must export a
new public symbol "PyModuleCreate_modulename", where "modulename" is the
name of the shared library. This mimics the previous naming convention for
the "PyInit_modulename" function.
This symbol must resolve to a C function with the following signature::
PyObject* (*PyModuleTypeCreateFunction)(PyObject* module_spec)
The "module_spec" argument receives a "ModuleSpec" instance, as defined in
PEP 4XX (FIXME). (All names are obviously up for debate and bike-shedding
at this point.)
When called, this function must create and return a type object, either a
Python class or an extension type that is allocated on the heap. This type
will be instantiated as module instance by the importer.
There is no requirement for this type to be exactly or a subtype of
types.ModuleType. Any type can be returned. This follows the current
support for allowing arbitrary objects in sys.modules and makes it easier
for extension modules to define a type that exactly matches their needs for
holding module state.
The constructor of this type must have the following signature::
def __init__(self, module_spec):
The "module_spec" argument receives the same object as the one passed into
the module type creation function.
Implementation
==============
XXX - not started
Reloading and Sub-Interpreters
==============================
To "reload" an extension module, the module create function is executed
again and returns a new module type. This type is then instantiated as by
the original module loader and replaces the previous entry in sys.modules.
Once the last references to the previous module and its type are gone, both
will be subject to normal garbage collection.
Sub-interpreter support is an inherent property of the design. During
import in the sub-interpreter, the module create function is executed
and returns a new module type that is local to the sub-interpreter. Both
the type and its module instance are subject to garbage collection in the
sub-interpreter.
Open questions
==============
It is not immediately obvious how extensions should be handled that want to
register more than one module in their module init function, e.g. compiled
packages. One possibility would be to leave the setup to the user, who
would have to know all FQMNs anyway in this case (or could construct them
from the module spec of the current module), although not the import
file path. A C-API could be provided to register new module types in the
current interpreter, given a user provided ModuleSpec.
There is no inherent requirement for the module creation function to
actually return a type. It could return a arbitrary callable that creates a
'modulish' object when called. Should there be a type check in place that
makes sure that what it returns is a type? I don't currently see a need for
this.
Copyright
=========
This document has been placed in the public domain.
I recently committed a fix for unicodeobject.c so that the %d, %i, and %u format specifiers always output values
(otherwise, in subclasses, the str() was used instead).
Should this be fixed in 3.3 as well?
What guidelines determine when a bug is fixed in previous versions?
--
~Ethan~
Part of the fix for issue #18693 is to fix inspect to look in the metaclass for class attributes
(http://bugs.python.org/issue18929).
In inspect.py in function get_mro() we can either add the metaclass unconditionally, or only if it is not 'type'.
If we add unconditionally, then help() adds the following:
class A(builtins.object)
| Hello and goodbye
+ |
+ | Method resolution order:
+ | A
+ | builtins.object
+ | builtins.type
|
| Methods defined here:
Do we want that, or should we just add the metaclass if it is not 'type'?
--
~Ethan~
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
With importlib and other recent changes, what are the current details
of "import lock"?. That is, the lock/locks held when Python code does
"import", specially in the case of multithreading. Is that documented
anywhere?
Thanks!
- --
Jesús Cea Avión _/_/ _/_/_/ _/_/_/
jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/
Twitter: @jcea _/_/ _/_/ _/_/_/_/_/
jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUinv7Jlgi5GaxT1NAQLX7QP/SBNuy869xSKRVNlWmAiDLpZOR9IHm77K
fy+AG/aJDpk4BMcm+KNoHZyUtJJs2kndfz+OVTeU/mmlr+aXu/wA1yNB1HAjAwWO
4ZWyaN9WLrhv8xXomUlT8iqikc/9pEspgQbJAljUvMLVouixIdxga3IqtaT3FQZh
wkV9v04G4OA=
=P3PI
-----END PGP SIGNATURE-----
Hello,
I was looking at the possibility of replacing the SEEK_* constants by
IntEnums, and the first thing that catches attention is that these
constants are defined in both Lib/os.py and Lib/io.py; both places also
recently started supporting SEEK_HOLE and SEEK_DATA (though here io refers
to os.SEEK_HOLE and os.SEEK_DATA).
Additional data points: other modules take these constants as arguments -
* mmap: directs to use os.SEEK_*
* chunk and fcntk: spell out the numeric values.
os seems to import io in some functions; can this be done always? If yes,
we can just define the constants once and os.SEEK_* will alias io.SEEK_*?
The other way (io taking from os) is also a possibility (maybe the
preferred one because io already refers to os.SEEK_HOLE/DATA, at least in
the documentation).
Any ideas and suggestions are welcome,
Eli
Hi All,
Continuous testing is a wonderful thing when it comes to finding weird
edge case problems, like this one:
http://jenkins.simplistix.co.uk/job/testfixtures-tox/COMPONENTS=zc,PYTHON=3…
File
"C:\Jenkins\workspace\testfixtures-tox\e8666d4e\.tox\3.3-zc\lib\site-packages\testfixtures\tempdirectory.py",
line 323, in __exit__
self.cleanup()
File
"C:\Jenkins\workspace\testfixtures-tox\e8666d4e\.tox\3.3-zc\lib\site-packages\testfixtures\tempdirectory.py",
line 78, in cleanup
rmtree(self.path)
File
"C:\Jenkins\workspace\testfixtures-tox\e8666d4e\.tox\3.3-zc\lib\shutil.py",
line 460, in rmtree
return _rmtree_unsafe(path, onerror)
File
"C:\Jenkins\workspace\testfixtures-tox\e8666d4e\.tox\3.3-zc\lib\shutil.py",
line 362, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File
"C:\Jenkins\workspace\testfixtures-tox\e8666d4e\.tox\3.3-zc\lib\shutil.py",
line 371, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File
"C:\Jenkins\workspace\testfixtures-tox\e8666d4e\.tox\3.3-zc\lib\shutil.py",
line 369, in _rmtree_unsafe
os.rmdir(path)
OSError: [WinError 145] The directory is not empty:
'c:\\users\\jenkins\\appdata\\local\\temp\\tmpkeg4d7\\a'
I'm 99% certain my code is correct here, the only place I open files for
writing in that directory is here:
https://github.com/Simplistix/testfixtures/blob/master/testfixtures/tempdir…
So, from my perspective, I'm either looking at a bug in shutil.rmtree
(which would be trying to delete a directory before deleting its content
or failing to delete a file but ignoring an error) or the file object,
when being used as a context manager, going through __exit__ without
closing the file and releasing the handle.
This happens very infrequently, the OS is Windows 7 and the filesystem
is NTFS, if that helps...
Any ideas?
Chris
--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
Hi,
Thanks to the PEP 445, it becomes possible to trace easily memory
allocations. I propose to add a new tracemalloc module computing the
memory usage per file and per line number. It has also a private
method to retrieve the location (filename and line number) of a memory
allocation of an object.
tracemalloc is different than Heapy or PySizer because it is focused
on the location of a memory allocation rather that the object type or
object content.
I have an implementation of the module for Python 2.5-3.4, but it
requires to patch and recompile Python:
https://pypi.python.org/pypi/pytracemalloc
My proposed implementation for Python 3.4 is different:
* reuse the PEP 445 to hook memory allocators
* use a simple C implementation of an hash table called "cfuhash"
(coming from the libcfu project, BSD license) instead of depending on
the glib library. I simplified and adapted cfuhash for my usage
* no enable() / disable() function: tracemalloc can only be enabled
before startup by setting PYTHONTRACEMALLOC=1 environment variable
* traces (size of the memory block, Python filename, Python line
number) are stored directly in the memory block, not in a separated
hash table
I chose PYTHONTRACEMALLOC env var instead of enable()/disable()
functions to be able to really trace *all* memory allocated by Python,
especially memory allocated at startup, during Python initialization.
The (high-level) API should be reviewed and discussed. The most
interesting part is to take "snapshots" and compare snapshots. The
module can load snapshots from disk and compare them later for deeper
analysis (ex: ignore some files).
For the documentation, see the following page:
https://pypi.python.org/pypi/pytracemalloc
I created the following issue to track the implementation:
http://bugs.python.org/issue18874
The implementation:
http://hg.python.org/features/tracemalloc
* * *
I also created a "pyfailmalloc" project based on the PEP 445 to inject
MemoryError exceptions. I used this module to check if Python handles
correctly memory allocation failures. The answer is no, I fixed many
bugs (see issue #18408).
Project homepage:
https://bitbucket.org/haypo/pyfailmalloc
Charles-François Natali and Serhiy Storchaka asked me to add this
module somewhere in Python 3.4: "how about adding pyfailmalloc to the
main repo (maybe under Tools), with a script making it easy to run the
tests suite with it enabled?"
What is the best place for such module? Add it to Modules/ directory
but as a private module: "_failmalloc"?
* * *
Example of tracemalloc output (it is more readable with colors, try in
a terminal). The first top is sorted by total size, whereas the second
top is sorted (automatically) with the size difference. You can see
for example that the linecache module likes caching data (1.5 MB after
10 seconds of tests).
$ PYTHONTRACEMALLOC=1 ./python -m test
...
== CPython 3.4.0a1+ (default:2ce9e5f6b47c+, Aug 29 2013, 02:03:02)
[GCC 4.7.2 20121109 (Red Hat 4.7.2-8)]
== Linux-3.9.4-200.fc18.x86_64-x86_64-with-fedora-18-Spherical_Cow
little-endian
== /home/haypo/prog/python/tracemalloc/build/test_python_11087
...
[ 1/380] test_grammar
[ 2/380] test_opcodes
[ 3/380] test_dict
[ 4/380] test_builtin
[ 5/380] test_exceptions
[ 6/380] test_types
[ 7/380] test_unittest
2013-08-29 02:06:22: Top 25 allocations per file and line
#1: <frozen importlib._bootstrap>:704: size=5 MiB, count=56227, average=105 B
#2: .../tracemalloc/Lib/linecache.py:127: size=1004 KiB, count=8706,
average=118 B
#3: .../Lib/unittest/mock.py:1764: size=895 KiB, count=7841, average=116 B
#4: .../Lib/unittest/mock.py:1805: size=817 KiB, count=15101, average=55 B
#5: .../Lib/test/test_dict.py:35: size=768 KiB, count=8, average=96 KiB
#6: <frozen importlib._bootstrap>:274: size=703 KiB, count=4604, average=156 B
#7: ???:?: size=511 KiB, count=4445, average=117 B
#8: .../Lib/unittest/mock.py:350: size=370 KiB, count=1227, average=308 B
#9: .../Lib/unittest/case.py:306: size=343 KiB, count=1390, average=253 B
#10: .../Lib/unittest/case.py:496: size=330 KiB, count=650, average=521 B
#11: .../Lib/unittest/case.py:327: size=291 KiB, count=717, average=416 B
#12: .../Lib/collections/__init__.py:368: size=239 KiB, count=2170,
average=113 B
#13: .../Lib/test/test_grammar.py:132: size=195 KiB, count=1250, average=159 B
#14: .../Lib/unittest/mock.py:379: size=118 KiB, count=152, average=800 B
#15: .../tracemalloc/Lib/contextlib.py:37: size=102 KiB, count=672,
average=156 B
#16: <frozen importlib._bootstrap>:1430: size=91 KiB, count=1193, average=78 B
#17: .../tracemalloc/Lib/inspect.py:1399: size=79 KiB, count=104, average=784 B
#18: .../tracemalloc/Lib/abc.py:133: size=77 KiB, count=275, average=289 B
#19: .../Lib/unittest/case.py:43: size=73 KiB, count=593, average=127 B
#20: .../Lib/unittest/mock.py:491: size=67 KiB, count=153, average=450 B
#21: <frozen importlib._bootstrap>:1438: size=64 KiB, count=20, average=3321 B
#22: .../Lib/unittest/case.py:535: size=56 KiB, count=76, average=766 B
#23: .../tracemalloc/Lib/sre_compile.py:508: size=54 KiB, count=115,
average=485 B
#24: .../Lib/unittest/case.py:300: size=48 KiB, count=616, average=80 B
#25: .../Lib/test/regrtest.py:1207: size=48 KiB, count=2, average=24 KiB
7333 more: size=4991 KiB, count=28051, average=182 B
Total Python memory: size=17 MiB, count=136358, average=136 B
Total process memory: size=42 MiB (ignore tracemalloc: 22 KiB)
[ 8/380] test_doctest
[ 9/380] test_doctest2
[ 10/380] test_support
[ 11/380] test___all__
2013-08-29 02:08:44: Top 25 allocations per file and line (compared to
2013-08-29 02:08:39)
#1: <frozen importlib._bootstrap>:704: size=8 MiB (+2853 KiB),
count=80879 (+24652), average=109 B
#2: .../tracemalloc/Lib/linecache.py:127: size=1562 KiB (+557 KiB),
count=13669 (+4964), average=117 B
#3: <frozen importlib._bootstrap>:274: size=955 KiB (+252 KiB),
count=6415 (+1811), average=152 B
#4: .../Lib/collections/__init__.py:368: size=333 KiB (+93 KiB),
count=3136 (+966), average=108 B
#5: .../tracemalloc/Lib/abc.py:133: size=148 KiB (+71 KiB), count=483
(+211), average=314 B
#6: .../Lib/urllib/parse.py:476: size=71 KiB (+71 KiB), count=969
(+969), average=75 B
#7: .../tracemalloc/Lib/base64.py:143: size=59 KiB (+59 KiB),
count=1025 (+1025), average=59 B
#8: .../tracemalloc/Lib/doctest.py:1283: size=56 KiB (+56 KiB),
count=507 (+507), average=113 B
#9: .../tracemalloc/Lib/sre_compile.py:508: size=89 KiB (+36 KiB),
count=199 (+86), average=460 B
#10: <frozen importlib._bootstrap>:53: size=67 KiB (+32 KiB),
count=505 (+242), average=136 B
#11: <frozen importlib._bootstrap>:1048: size=61 KiB (+27 KiB),
count=332 (+138), average=188 B
#12: .../Lib/unittest/case.py:496: size=351 KiB (+25 KiB), count=688
(+48), average=522 B
#13: .../tracemalloc/Lib/_weakrefset.py:38: size=55 KiB (+24 KiB),
count=521 (+236), average=109 B
#14: .../Lib/email/quoprimime.py:56: size=24 KiB (+24 KiB), count=190
(+190), average=132 B
#15: .../Lib/email/quoprimime.py:57: size=24 KiB (+24 KiB), count=1 (+1)
#16: .../test/support/__init__.py:1055: size=24 KiB (+24 KiB), count=1 (+1)
#17: .../Lib/test/test___all__.py:48: size=23 KiB (+23 KiB), count=283
(+283), average=84 B
#18: .../tracemalloc/Lib/_weakrefset.py:37: size=60 KiB (+22 KiB),
count=438 (+174), average=140 B
#19: .../tracemalloc/Lib/sre_parse.py:73: size=48 KiB (+20 KiB),
count=263 (+114), average=189 B
#20: <string>:5: size=61 KiB (+18 KiB), count=173 (+57), average=364 B
#21: .../Lib/test/test___all__.py:34: size=16 KiB (+16 KiB), count=164
(+164), average=104 B
#22: .../Lib/unittest/mock.py:491: size=79 KiB (+16 KiB), count=145
(+3), average=560 B
#23: .../Lib/collections/__init__.py:362: size=50 KiB (+16 KiB),
count=23 (+7), average=2255 B
#24: .../Lib/test/test___all__.py:59: size=13 KiB (+13 KiB), count=165
(+165), average=84 B
#25: .../tracemalloc/Lib/doctest.py:1291: size=13 KiB (+13 KiB),
count=170 (+170), average=81 B
10788 more: size=7 MiB (-830 KiB), count=36291 (-16379), average=220 B
Total Python memory: size=20 MiB (+3567 KiB), count=147635 (+20805),
average=143 B
Total process memory: size=49 MiB (+7 MiB) (ignore tracemalloc: 1669 KiB)
Victor
If you've read my blog (eg: on planet python), you will be aware that
I dedicated August to full time email package development. At the
beginning of the month I worked out a design proposal for the remaining
API additions to the email package, dealing with handling message bodies
in a more natural way. I posted this to the email-sig, and got...well,
no objections. Barry Warsaw did review it, and told me he had no issues
with the overall design, but also had no time for a detailed review.
Since one way to see if a design holds together is to document and code
it, I decided to go ahead and do so. This resulted in a number of small
tweaks, but no major changes.
I have at this point completed the coding. You can view the whole
patch at:
http://bugs.python.org/issue18891
which also links to three layered patches that I posted as I went along,
if you prefer somewhat smaller patches.
I think it would be great if I could check this in for alpha2. Since it
is going in as an addition to the existing provisional code, the level
of review required is not as high as for non-provisional code, I think.
But I would certainly appreciate review from anyone so moved, since I
haven't gotten any yet.
Of course, if there is serious bikeshedding about the API, I won't
make alpha2, but that's fine.
The longer term goal, by the way, is to move all of this out of
provisional status for 3.5.
This code finishes the planned API additions for the email package
to bring it fully into the world of Python3 and unicode. It does not
"fix" the deep internals, which could be a future development direction
(but probably only after the "old" API has been retired, which will take
a while). But it does make it so that you can use the email package
without having to be a MIME expert. (You can't get away with *no* MIME
knowledge, but you no longer have to fuss with the details of the syntax.)
To give you the flavor of how the entire new provisional API plays
together, here's how you can build a complete message in your application:
from email.message import MIMEMessage
from email.headerregistry import Address
fullmsg = MIMEMessage()
fullmsg['To'] = Address('Foö Bar', 'fbar(a)example.com')
fullmsg['From'] = "mè <me(a)example.com>"
fullmsg['Subject'] = "j'ai un problème de python."
fullmsg.set_content("et la il est monté sur moi et il commence"
" a m'étouffer.")
htmlmsg = MIMEMessage()
htmlmsg.set_content("<p>et la il est monté sur moi et il commence"
" a m'étouffer.</p><img src='image1' />",
subtype='html')
with open('python.jpg', 'rb') as python:
htmlmsg.add_related(python.read(), 'image', 'jpg', cid='image1'
disposition='inline')
fullmsg.make_alternative()
fullmsg.attach(htmlmsg)
with open('police-report.txt') as report:
fullmsg.add_attachment(report.read(), filename='pölice-report.txt',
params=dict(wrap='flow'), headers=(
'X-Secret-Level: top',
'X-Authorization: Monty'))
Which results in:
>>> for line in bytes(fullmsg).splitlines():
>>> print(line)
b'To: =?utf-8?q?Fo=C3=B6?= Bar <fbar(a)example.com>'
b'From: =?utf-8?q?m=C3=A8?= <me(a)example.com>'
b"Subject: j'ai un =?utf-8?q?probl=C3=A8me?= de python."
b'MIME-Version: 1.0'
b'Content-Type: multipart/mixed; boundary="===============1710006838=="'
b''
b'--===============1710006838=='
b'Content-Type: multipart/alternative; boundary="===============1811969196=="'
b''
b'--===============1811969196=='
b'Content-Type: text/plain; charset="utf-8"'
b'Content-Transfer-Encoding: 8bit'
b''
b"et la il est mont\xc3\xa9 sur moi et il commence a m'\xc3\xa9touffer."
b''
b'--===============1811969196=='
b'MIME-Version: 1.0'
b'Content-Type: multipart/related; boundary="===============1469657937=="'
b''
b'--===============1469657937=='
b'Content-Type: text/html; charset="utf-8"'
b'Content-Transfer-Encoding: quoted-printable'
b''
b"<p>et la il est mont=C3=A9 sur moi et il commence a m'=C3=A9touffer.</p><img ="
b"src=3D'image1' />"
b''
b'--===============1469657937=='
b'MIME-Version: 1.0'
b'Content-Type: image/jpg'
b'Content-Transfer-Encoding: base64'
b'Content-Disposition: inline'
b'Content-ID: image1'
b''
b'ZmFrZSBpbWFnZSBkYXRhCg=='
b''
b'--===============1469657937==--'
b'--===============1811969196==--'
b'--===============1710006838=='
b'MIME-Version: 1.0'
b'X-Secret-Level: top'
b'X-Authorization: Monty'
b'Content-Transfer-Encoding: 7bit'
b'Content-Disposition: attachment; filename*=utf-8''p%C3%B6lice-report.txt"
b'Content-Type: text/plain; charset="utf-8"; wrap="flow"'
b''
b'il est sorti de son vivarium.'
b''
b'--===============1710006838==--'
If you've used the email package enough to be annoyed by it, you may
notice that there are some nice things going on there, such as using
CTE 8bit for the text part by default, and quoted-printable instead of
base64 for utf8 when the lines are long enough to need wrapping.
(Hmm. Looking at that I see I didn't fully fix a bug I had meant to fix:
some of the parts have a MIME-Version header that don't need it.)
All input strings are unicode, and the library takes care of doing
whatever encoding is required. When you pull data out of a parsed
message, you get unicode, without having to worry about how to decode
it yourself.
On the parsing side, after the above message has been parsed into a
message object, we can do:
>>> print(fullmsg['to'], fullmsg['from'])
Foö Bar <"fbar(a)example.com"> mè <me(a)example.com>
>>> print(fullmsg['subject'])
j'ai un problème de python.
>>> print(fullmsg['to'].addresses[0].display_name)
Foö Bar
>>> print(fullmsg.get_body(('plain',)).get_content())
et la il est monté sur moi et il commence a m'étouffer.
>>> for part in fullmsg.get_body().iter_parts():
... print(part.get_content())
<p>et la il est monté sur moi et il commence a m'étouffer.</p><img src='image1' />
b'fake image data\n'
>>> for attachment in fullmsg.iter_attachments():
... print(attachment.get_content())
... print(attachment['Content-Type'].params())
il est sorti de son vivarium.
{'charset': 'utf-8', 'wrap': 'flow'}
Of course, in a real program you'd actually be checking the mime types
via get_content_type() and friends before getting the content and doing
anything with it.
Please read the new contentmanager module docs in the patch for
full details of the content management part of the above API (and the
headerregistry docs if you want to review the (new in 3.3) header parsing
part of the above API).
Feedback welcome, here or on the issue.
--David
PS: python jokes courtesy of someone doing a drive-by on #python-dev the other
day.
Hello Pythonistas,
Python 2.6.9 is the last planned release of the 2.6.x series. This will be a
security-only source-only release. It is currently scheduled for October
2013, and after this Python 2.6 will have reached its end-of-life and the
branch will be retired.
http://www.python.org/dev/peps/pep-0361/
I would like to release 2.6.9rc1 on Monday, September 30th, and 2.6.9 final on
Monday, October 28th. I've added both dates to the Python calendar.
Here are the list of candidates still to be fixed for 2.6.9:
- 18747 - Re-seed OpenSSL's PRNG after fork
- 16037 - httplib: header parsing is not delimited
- 16038 - ftplib: unlimited readline() from connection
- 16039 - imaplib: unlimited readline() from connection
- 16040 - nntplib: unlimited readline() from connection
- 16041 - poplib: unlimited readline() from connection
- 16042 - smtplib: unlimited readline() from connection
- 16043 - xmlrpc: gzip_decode has unlimited read()
These were the ones I previously had on my list, and I've now marked these all
as release blockers for 2.6.9... for now.
If you know of any others that I should be aware of, please let me know.
If you can contribute to 2.6.9 by reviewing, testing, developing, or
commenting, that would be greatly appreciated. I will be spending some time
triaging these and any other issues that get identified as possible 2.6.9
candidates.
If you have any questions regarding 2.6.9, please contact me via mailing list
or IRC.
Cheers,
-Barry
