Mailman 3 June 2009 - Python-Dev

ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS
by Filippo Battaglia 25 Jun '09

25 Jun '09

I need help with an implementation of your interpreter under Nanodesktop PSPE/PSP. I'm working to a version of Stackless Python interpreter (2.5.2) called ndPython. It is able to work under PSPE and PSP and it shall be the most powerful interpreter ever realized before on these platforms. I need to know something about the C functions that are recalled by the interpreter when it executes a .pyc file. Our version of ndPython is very slow in execution respect to Carlos's StackLess Python (another product for PSP). We believe that the trouble is in any routine of our Nanodesktop libc that could be a bottleneck. But we don't know which can be the interested routine (string ? memory allocation ?) Can you tell me which modules provide to decode and execute a .pyc file ? The number of modules that are statically linked in the interpreter can affect its performance ? (for example, for a larger number of strcmp between the keywords). The execution of a .pyc file is a memory allocation intensive task ? I've already enable the PYMEM option and I've see that the speed is improved (but Carlos's StackLess Python remains faster). We've done these tests: http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20 from ndpsp import * x = time.clock () for i in range(1000): pass print >> clock, "1.", time.clock()-t We've the results: stackless: 0.0037279 ndPython: 0.1015625 We don't understand why ndPython is so slower. Please, help us. The product is complete, but we cannot release cause this problem. Thank you in advance. Filippo Battaglia -- Caselle da 1GB, trasmetti allegati fino a 3GB e in piu' IMAP, POP3 e SMTP autenticato? GRATIS solo con Email.it http://www.email.it/f Sponsor: Vieni in vacanza nei Riccione Family Hotels! Ti aspettano servizi per bambini, mini club e tanto divertimento! Prenota on-line la tua vacanza! Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=9206&d=24-6

5 5

Syntax for parsing tuples/lists into C arrays
by Yingjie Lan 25 Jun '09

25 Jun '09

The current C API does not support directly parsing into C arrays. Parsing tuples into C arrays can be greatly facilitated with the introduction of a natural extension of the current parsing syntax. I'd like to propose a syntax extension to facilitate that kind of tasks, and would appreciate your feedbacks. The extension comes from the way strings are parsed in current C API: { const char * s; int i; PyArg_ParseTyple(args, "s#", &s, &i); } Consider for a moment the string as a sequence of char's, which by the example above is parsed into an array of char's. Note that in python, a tuple of some type (e.g. int's), is also a sequence of that type, This perspective provides a natural way to extend the the use of "#" symbol -- what is missing is how to signal a tuple as a sequence, without interfering with current fixed-length tuple parsing syntax. To signal a tuple as a sequence of a given type of objects, "(type:)" can be used. For example, "(i:)" signals a tuple of integers as array of integers. The ":" is chosen to denote possible repetition of the whole pattern inside the parentheses. Now consider 3 different scenarios of parsing tuples into arrays. Scenario (1) Unkown size. Here are some examples: { //parsing a tuple and its size int i, *t; PyArg_ParseTuple(args, "(i:)#", &t, &i); } { //parsing a matrix of i rows and j columns: int i, j, **t; PyArg_ParseTuple(args, "((i:):)##", &t, &i, &j); } Here is an informal explanation of "((i:):)##": the '#' can be seen as a peeling operator, which peels away the outer most "(...:)" that denote a "free sequence" (i.e. sequence of unknown size) with some pattern "...". For types like 's' or its similar types, pretend as if there is an invisible pair of parentheses around it. For example, "(s:)##" matches a tuple of strings of some fixed length. Here the first '#' peels away the parentheses, leaving only "s#", which is equivalent to "(*char*:)#" in concept, where "*char*" denote the character type (not exist in Python). As the second '#' is outside the tuple, it is shared by all strings inside, so they _must_ have the same length. If there is nothing to peel for a '#', it is considered a dangling '#', an error. Now consider some more interesting examples: { int i, *t, *v; PyArg_ParseTuple(args, "(ii:)#", &t, &v, &i); //args=(1,2,3,4,5,6) //t={1,3,5} //v={2,4,6} //i=3 } In that example, a pattern is provided inside the sequence "(ii:)", which can repeat itself multiple times. The repeated times is extracted by the '#' symbol. Below is another example: { int i,j, *t, **v; PyArg_ParseTuple(args, "(i(i:):)##", &t, &v, &i, &j); //args=(1,(3,),5,(7,)) //t={1,5} //v={{3},{7}} //i=2, j=1 } And another one: { int i,j, **t, **v; PyArg_ParseTuple(args, "((i:)(i:):)##", &t, &v, &i, &j); //args=((1,),(3,),(5,),(7,)) //t={{1},{5}} //v={{3},{7}} //i=2, j=1 } Note the second '#' is applied to both the free sequences inside the outer tuple. This indicates that all the inside tuples must be of the same length. What if they have their own lengths? We provide a separate matching '#' for each inside tuple: { int i,j,k, **t, **v; PyArg_ParseTuple(args, "((i:)(i:):)#(##)", &t, &v, &i, &j,&k); //args=((1,),(3,4),(5,),(7,8)) //t={{1},{5}} //v={{3,4},{7,8}} //i=2, j=1, k=2 } The syntax can also deal with more complicated situations, such as an irregular 2-dimensional array, where each row has a different row length: { int i, *j, **t; PyArg_ParseTuple(args, "((i:)#:)#", &t, &j, &i); for(a=0; a) //t={{{1,2},{1,2}},{{1,2}}} //i=2, k=2 //j={2,1} } This syntax can be used for parsing a tuple of strings of different lengths: { int i, *j; const char ** s; PyArg_ParseTuple(args, "(s#:)#", &s, &j, &i); //args=("ab", "cde") //s={"ab", "cde"} //j={2, 3} //i=2 } Scenario (2) If you know the size of a tuple before hand, use this syntax: "(*type*integer)", meaning there are exactly 'integer' number of *type* objects in the tuple. { int *t; // PyArg_ParseTuple(args, "(i5)", &t); //args = (1,2,3,4,5) //t = {1,2,3,4,5} } A complicated case: { int **t; PyArg_ParseTuple(args, "((i5)2)", &t); //args=((1,2,3,4,5),(5,6,7,8,9)) //t={{1,2,3,4,5},{5,6,7,8,9}} } Another complicated case: { int **t, **v, i; PyArg_ParseTuple(args, "(i3i2:)#", &t,&v,&i); //args=(1,2,3,4,5,6,7,8,9,10) //t={{1,2,3},{6,7,8}} //v={{4,5},{9,10}} //i=2 } One more thing: the difference between "(ii)" and "(i2)" is that the former yields two single integers, while the latter gives an array of length two. Scenario (3) Concatenating arrays while parsing. Sometimes you would like to have a matrix parsed into a single array. We can use a "(+" to signal that the arrays parsed from the elements in that tuple should be concatenated into a single array. { int i,j, *t; PyArg_ParseTuple(args, "(+(i))##", &t, &i, &j); //args=((1,2,3),(4,5,6)) //t={1,2,3,4,5,6} //i=2, j=3 } Another example: { int i, *t; PyArg_ParseTuple(args, "(+(i2))#", &t, &i); } Still another example, with strings: { int i, *j; const char * t; //string PyArg_ParseTuple(args, "(+(+s#2))#", &t, &j, &i); //args=(("ab","c"),("d","efg")) //t="abcdefg" //i=2 ////notice the size arrays are concatenated similarly //j={2,1,1,3} //without '+': j={{2,1},{1,3}} const char ** sa; //The '*' in place of '#' to discard the size info PyArg_ParseTuple(args, "(+(s*2))", &sa); //args=(("ab","c"),("d","efg")) //sa = {"ab","c","d","efg"} PyArg_ParseTuple(args, "((+s*2))", &sa); //args=(("ab","c"),("d","efg")) //sa = { "abc", "defg" } } And finally, an example with multi-item pattern: { int *t, *v, i; PyArg_ParseTuple(args, "(+i1i2:)#", &t, &v, &i); //args=(1,2,3,4,5,6) //t={1,4} //v={2,3,5,6} //i=2 } Epilogue: This syntax can be applied to lists: just use "[...:]" instead of "(...:)". It can also be used for building tuples or lists out of arrays. That's it. Comments? Yingjie

3 2

trace module options.
by Edward Peschko 25 Jun '09

25 Jun '09

All, I've been looking at the trace module, and although it looks useful, I'm surprised that there aren't a couple of features that I would have thought would be fairly basic. So, does trace support (for the --trace option): - indentation tracking stacklevel (where each function is prefixed by tabs equal to the number of stacklevels deep in the program) - output to something other than sys.stdout (eg. output to a file specified either by environmental variable or by parameter). - mult-threaded programs going to multiple output handles, especially in light of the above - fully qualified python modules in path: (eg: /path/to/module/my_module.py(1): print "HERE" instead of my_module.py(1): print "HERE". Ultimately, I'd like to be able to look at two runs of a program and be able to pinpoint the very first difference between thembased on the output of their trace runs. As it stands, I really can't do this. Of course I could program the above, but I was hoping to avoid duplicated effort if someone has already programmed options like this.. Ed

2 2

Python for Windows??
by David H. Burns 25 Jun '09

25 Jun '09

I downloaded what claims to be Python for Windows (3.01). The tutorial brags a lot about how easy it is to learn, but the tutorials and instruction seem to be for a Linux or Unix version. There are three executable programs in the Python directory and no indication which should be used to start Python. One opens a Dos-like window presumably for "command-line" entry. I can't make anything of it. I did a straight install on a XP system. Any help would be appreciated. Have a good day, David H. Burns

4 3

Re: [Python-Dev] PEP 376
by P.J. Eby 24 Jun '09

24 Jun '09

At 10:38 AM 6/23/2009 +0200, Tarek Ziadé wrote: >What about using the same names used in Python's site module: >"sitedir" is the name used for >a directory we named DistributionDirectory. No, a site dir is a Python-defined directory for site-installed packages, and/or a directory where .pth files are processed. Wrong connotations entirely, since packages may be installed to other directories, and typically are in e.g. shared hosting environments. DistributionDirectory is fine by me. DistributionDirectories sounds like what setuptools calls an Environment.

1 0

Adding syntax for units of measure
by Tobias C. Rittweiler 24 Jun '09

24 Jun '09

Hi! Has anyone added special syntax to allow writing numeric literals with physical units? So you can write 12m + 34cm, and would get 12.34m. My question is how would you modify the BNF the most sensible way to allow for this? The above example is simple, but think of 42 km/h. (For my purposes, modifying the BNF is perfectly reasonable, but if you can depict a good, and convenient!, way that would not result in modifying it, I'd like to hear it, too.) Thanks in advance for your input, -T.

5 4

Python 3.0.1 and mingw
by Vincent R. 23 Jun '09

23 Jun '09

Hi, I wanted to know if you have some patch to compile python 3.x on mingw platform because I found some but doesn't work very well : > make gcc -o python.exe \ Modules/python.o \ libpython3.0.a -lm Could not find platform independent libraries <prefix> Could not find platform dependent libraries <exec_prefix> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] Fatal Python error: Py_Initialize: can't initialize sys standard streams ImportError: No module named encodings.utf_8 I have some questions about posixmodule.c, config.c and makesetup, I can see in posixmodule that you define a INITFUNC like this : #if (defined(_MSC_VER) || defined(__WATCOMC__) || defined(__BORLANDC__)) && !defined(__QNX__) #define INITFUNC PyInit_nt #define MODNAME "nt" #elif defined(PYOS_OS2) #define INITFUNC PyInit_os2 #define MODNAME "os2" #else #define INITFUNC PyInit_posix #define MODNAME "posix" #endif So first I tried to add || defined(____MINGW32____) to declare a PyInit_nt but after config.c was still using PyInit_posix. How does makesetup choose to include one function or another ? So finally python is using PyInit_posix... Any idea why I got this compilation error ? And after

2 1

Re: [Python-Dev] PEP 376
by P.J. Eby 23 Jun '09

23 Jun '09

At 05:42 PM 6/22/2009 +0200, Tarek Ziadé wrote: >Wouldn't it be better to use the native line terminator on the current >platform? (someone might want to edit or at least view the file) > > >Good idea, I'll change that, > As long as the file is always *read* with "U" mode, so that you can't mess it up, especially if the install is to a directory shared between platforms. >The idea of this API is to find out of a distribution "owns" a file, >e.g. is the only distribution >that uses it, so it can be safely removed. This could equally well be done by ``owners(path)``, returning a sequence of zero or more items. Any length <> 1 means the file can't be safely removed. Meanwhile, having the data about all the owners of a file would also be useful for tools that just want to inspect a directory's contents, for example, or to detect conflicts and overwrites.

2 1

final release procedure
by Benjamin Peterson 22 Jun '09

22 Jun '09

Hi everyone, We're almost to the finish line on 3.1! Assuming that the last release blocker [1] gets ironed out and no more rear their ugly heads, I will go ahead. I'm going to freeze and tag the tree on Friday at about 15:00 UTC in order for the binaries to be built and uploaded. The release announcement should be the next day. [1] http://bugs.python.org/issue6233 -- Regards, Benjamin

1 0

draft pep: backwards compatibility
by Benjamin Peterson 22 Jun '09

22 Jun '09

Backwards compatibility seems to be an issue that arises a lot here. I think we all have an idea of it is, but we need some hard place to point to. So here's my attempt: PEP: 387 Title: Backwards Compatibility Policy Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson <benjamin(a)python.org> Status: Draft Type: Process Content-Type: text/x-rst Created: 18-Jun-2009 Abstract ======== This PEP outlines Python's backwards compatibility policy. Rationale ========= As one of the most used programming languages today [#tiobe]_, the Python core language and its standard library play a critcal role in thousands of applications and libraries. This is fantastic; it is probably one of a language designer's most wishful dreams. However, it means the development team must be very careful not to break this existing 3rd party code with new releases. Backwards Compatibility Rules ============================= This policy applys to all public APIs. These include the C-API, the standard library, and the core language including syntax and operation as defined by the reference manual. This is the basic policy for backwards compatibility: * The behavior of an API *must* not change between any two consecutive releases. * A feature cannot be removed without notice between any two consecutive releases. * Addition of a feature which breaks 3rd party libraries or applications should have a large benefit to breakage ratio, and/or the incompatibility should be trival to fix in broken code. Making Incompatible Changes =========================== It's a fact: design mistakes happen. Thus it is important to be able to change APIs or remove misguided features. This is accomplished through a gradual process over several releases: 1. Discuss the change. Depending on the size of the incompatibility, this could be on the bug tracker, python-dev, python-list, or the appropriate SIG. A PEP or similar document may be written. Hopefully users of the affected API will pipe up to comment. 2. Add a warning [#warnings_]_. If behavior is changing, a the API may gain a new function or method to perform the new behavior; old usage should raise the warning. If an API is being removed, simply warn whenever it is entered. DeprecationWarning is the usual warning category to use, but PendingDeprecationWarning may be used in special cases were the old and new versions of the API will coexist for many releases. 3. Wait for a release. 4. See if there's any feedback. Users not involved in the original discussions may comment now after seeing the warning. Perhaps reconsider. 5. The behavior change or feature removal may now be made default or permanent in the next release. Remove the old version and warning. References ========== .. [#tiobe] TIOBE Programming Community Index http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html .. [#warnings] The warnings module http://docs.python.org/library/warnings.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

12 32