Mailman 3 September 2001 - Python-Dev

Progress on Psyco
by Armin Rigo Sept. 20, 2001

Sept. 20, 2001

Hello everybody, A short note about my progress on Psyco, the kind of compiler for Python. In its latest form it is a kind of just-in-time compiler, working at run-time. It consists of two parts: a language-independent "glue" part (which I more or less finished), and the inner loop of the Python interpreter rewritten in a special, resticted language. The "glue" part executes Python programs by running the interpreter, but not in the common way. The run-time data are split into what must be known to produce good processor code, and what need not be. Typically, we need to know at least which Python code to interpret before we can make good processor code that performs similarily; in the case of Python we will probably also need to know the type fields of the actual Python objects fed to the Python code; the processor code can be very efficiently optimized depending on the type. On the other hand, if the Python objects are, say, integers, then the same processor code can easily manipulate all possible integer values. The "glue" code is responsible for choosing which data goes into which category, and managing caches of the produced code. The Python-dependent part of the code either makes computations at compile-time (for compile-time-known values) or emits code to do it at run-time (for the rest). The choice between the two categories of data is not done statically; I mean, I didn't say "Python code and type fields are in category 1, the rest in category 2", which would amount to writing a JIT compiler with type-analysis. No; I let this separation be done based on the uses of the data. In two words, if I have a "switch" on data, or a call to function pointer, then knowing the data at compile time enables significant improvements, so I mark that data as "category 1"; as long as this does not occur data is in "category 2". This makes the approach quite general, I think. Experiment will be needed to know whether it performs reasonably well. Another interesting point to note is that this idea applies to any other interpreted language; all you need to do is rewrite the inner loop of the interpreter for the other language in a special way. Orthogonally, you can change the target processor by adapting parts of the "glue" code. More about it later, Armin.

1 0

copy, len and the like as 'object' methods?
by David Ascher Sept. 20, 2001

Sept. 20, 2001

I just came back from teaching Python on a cruise ship attended by mostly Perl folks. It was an interesting experience teaching Python with Randal in the audience =). One issue that came up is some of the lack of uniformity on what things are statements, methods and built-in functions. In the long-term version of the type/class unification work, things like int() become class constructors, which makes beautiful sense. The fact that int() calls __int__ methods fits nicely with type conversion mechanisms. However, there are a few things which still seem oddballish: copy.copy(), copy.deepcopy(), len() These basically call magic methods of their arguments (whether tp_slots or __methods__), and many languages implement them strictly as object methods. str() and repr() are a little weird -- I'm not sure which one will gain 'class constructor' status when the type/class unification work is done -- from the definition I'd say repr() should win, but the name is quite unfortunate given its new role... Guido, thoughts? Summary: Should copy, deepcopy and len be added as object methods? And if yes, how? Not all objects are copyable or measurable. Interfaces seem the right way to do this, but interfaces aren't in the plans so far that I know... What about a stringification method? --david

10 21

Parametrized codecs
by Andrew Kuchling Sept. 20, 2001

Sept. 20, 2001

Are there any codecs that can take parameters at initialization time? If not, how about supporting them? Here's a motivating example: import codecs f = codecs.open('output', 'wb', encoding='DES', # arguments to open() key = '...', mode = DES.ECB) f.write('This will be encrypted\n') f.close() For this to work, you'd need a few changes to codecs.py. Off the top of my head, I think StreamReader, StreamWriter, and StreamReaderWriter all need to grow **kwargs arguments for any additional arguments, as do codecs.open() and codecs.lookup(). Then someone could write a DES StreamReader/Streamwriter that took a key and feedback mode at creation time. (codecs.Codec instances are supposed to be stateless, right?) --amk

3 7

RE: [Python-Dev] Unaligned accesses in coercions, current CSV
by Mark.Favas＠csiro.au Sept. 20, 2001

Sept. 20, 2001

Yes, patch take 2 works just fine on Tru64 - thanks! MCF -----Original Message----- From: Guido van Rossum [mailto:guido@python.org] Sent: Thursday, 20 September 2001 1:01 PM To: Mark Favas Cc: python-dev(a)python.org Subject: Re: [Python-Dev] Unaligned accesses in coercions, current CSV > On a Compaq Alpha (Tru64 Unix, 4.0F), the following code (from > test_descr.py) produces unaligned access warnings when run: I think this is a duplicate of bug report #462848. See the second patch attached there and let me know if it fixes it! --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

11
by eleven11＠hushmail.com Sept. 20, 2001

Sept. 20, 2001

World Trade Center Disaster Facts The date of the attack 9/11 9+1+1=11 September 11th is the 254th day of the year: 2+5+4=11 After September 11th there are 111 days left to the end of the year. Twin Towers - standing side by side look like the number 11. The first plane to hit the towers was Flight 11. State of New York is the 11th state added to the union. New York City 11 letters The Pentagon 11 letters Afghanistan 11 letters Ranzi Yousef 11 letters (Convicted of orchestrating the 1993 attack on the WTC) September 11 11 letters Flight 11 had 92 on board 9+2=11 Flight 77 had 65 on board 6+5=11

1 0

Unaligned accesses in coercions, current CSV
by Mark Favas Sept. 20, 2001

Sept. 20, 2001

On a Compaq Alpha (Tru64 Unix, 4.0F), the following code (from test_descr.py) produces unaligned access warnings when run: class L(long): pass coerce(L(0), 0) coerce(L(0), 0L) coerce(0, L(0)) coerce(0L, L(0)) Unaligned access pid=5070 <python> va=0x1400c9266 pc=0x120031228 ra=0x120031218 inst=0xa6000000 Unaligned access pid=5070 <python> va=0x1400d9086 pc=0x120031228 ra=0x120031218 inst=0xa6000000 Unaligned access pid=5070 <python> va=0x1400c9266 pc=0x120031228 ra=0x120031218 inst=0xa6000000 Unaligned access pid=5070 <python> va=0x1400c9266 pc=0x120031228 ra=0x120031218 inst=0xa6000000 I'll try to chase this further, but work is a bit fraught at the moment... If you want me to log it as a bug, let me know. -- Mark Favas - m.favas(a)per.dem.csiro.au CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA

2 1

ModuleFinder patch
by Thomas Heller Sept. 19, 2001

Sept. 19, 2001

I've uploaded a freeze/ModuleFinder patch to sf, #462936. This patch adds two improvements: 1. ModuleFinder now keeps track of which module is imported by whom. 2. ModuleFinder, when instantiated with the new scan_extdeps=1 argument, tries to track dependencies of builtin and extension modules. I'd love to hear some comments especially on the last point, so I'll shortly explain how it works. ModuleFinder starts a separate python interpreter process with a command line of '-S -v -c "import <builtin_or_extension>"', pipes this through popen(), and scans the error-output for lines of the form 'import blabla # ....'. This is somewhat expensive, but it is the only way I've found so far to find out about these dependencies. Examples are: The multiarray extension needs the _numpy extension. Every extension using ExtensionClass needs this. cPickle needs copy_reg and other modules. Any other ideas? Regards, Thomas

1 0

why doesn't print pass unicode strings on to the file object?
by Martin von Loewis Sept. 19, 2001

Sept. 19, 2001

> is there any reason why "print" cannot pass unicode > strings on to the underlying write method? Mostly because there is no guarantee that every .write method will support Unicode objects. I see two options: either a stream might declare itself as supporting unicode on output (by, say, providing a unicode attribute), or all streams are required by BDFL pronouncement to accept Unicode objects. BTW, your wrapper example can be rewritten as import sys,codecs sys.stdout = codecs.lookup("iso-8859-1")[3](sys.stdout) I wish codecs.lookup returned a record with named fields, instead of a list, so I could write sys.stdout = codecs.lookup("iso-8859-1").writer(sys.stdout) (the other field names would be encode,decode, and reader). Regards, Martin

5 23

[development doc updates]
by Fred L. Drake Sept. 19, 2001

Sept. 19, 2001

The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Miscellaneous minor updates, including docs for the new codec interfaces.

1 0

Parametrized codecs
by Martin v. Loewis Sept. 19, 2001

Sept. 19, 2001

> Are there any codecs that can take parameters at initialization time? _codecs.utf_16_encode takes an additional parameter to request a certain endianness. This is not used in the codecs modules, though, since there are also functions for each endianness. > If not, how about supporting them? I think this would be up to the individual codecs to support; I'm -0 on the changes to codecs.open. >f = codecs.open('output', 'wb', encoding='DES', # arguments to open() > key = '...', mode = DES.ECB) Would this better be spelled f = codecs.open('output', 'wb', encoding='DES-ECB', # arguments to open() key = '...') To write it your way: Where did you get the DES module from? If the user has to specifically access the DES module, anyway, how about f = DES.open('output', 'wb', key = '...', mode = DES.ECB) > For this to work, you'd need a few changes to codecs.py. Off the > top of my head, I think StreamReader, StreamWriter, and > StreamReaderWriter all need to grow **kwargs arguments for any > additional arguments, Why is that? Shouldn't you instead provide a StreamReader etc. subclass that just accepts the additional arguments? Users could then write writer = codecs.lookup("DES")[3] f = writer(open('output', 'wb'), key="...", mode=DES.ECB) > as do codecs.open() and codecs.lookup() I cannot see at all why you want to pass arguments to lookup. Instead, wouldn't it be sufficient to pass them to the stream readers and codec functions returned? This also complicates caching of codecs, and probably the implementation of the codecs: how is lookup supposed to preserve the state in the tuple being returned? On arguments to open: mixing arguments that are consumed at different places (i.e. in .open, and in the reader/writer) makes me nervous. If we ever find the need for further standard arguments to codecs.open, it may break codecs that use these argument names for their own purposes. Regards, Martin

1 0