Mailman 3 January 2008 - Python-Dev

Changing PyAPI_DATA(type) into PyAPI_DATA(type, variablename)?
by Jukka.P.Laurila＠nokia.com 09 Jan '08

09 Jan '08

Hello, python-dev! One-liner summary: Would it be possible to change PyAPI_DATA(type) into PyAPI_DATA(type, variablename) to help portability to funny platforms? We've been working on porting the Python 2.5.1 core to the Symbian S60 smartphone platform. Unlike the previous 2.2.2 port, this time we're trying really hard to keep our changes clean so that they could some day be accepted into the main Python codebase. One problem we've encountered is that Symbian doesn't properly support exporting data from a DLL, and instead we must export a function that returns a pointer to the variable in question, and hide this from client applications using a macro: In a generated header, dataexports.h: #define Py_Verbose (*__DATAEXPORT_Py_Verbose()) extern __declspec(dllimport) int *__DATAEXPORT_Py_Verbose(); In a generated C file, dataexport.c: __declspec(dllexport) int *__DATAEXPORT_Py_Verbose() { return &Py_Verbose; } dataexports.h is included into Python.h when Py_BUILD_CORE isn't defined. Now, the problem is that we haven't figured out any way to define the PyAPI_DATA macro in its one-argument form so that this scheme would work, since when compiling client applications we'd like all the PyAPI_DATA declarations to expand to nothing, and get the declarations from dataexport.h instead. The best solution we've come up with is to change PyAPI_DATA into a two-argument form and change all declarations accordingly. Whether to export data with this mechanism or directly could be a pyconfig.h option, DATA_EXPORT_AS_FUNCTIONS (or HAVE_DATA_EXPORT?). Briefly the changes would touch: pyport.h: #ifdef Py_BUILD_CORE #define PyAPI_DATA(RTYPE,NAME) extern RTYPE NAME // data item not exported from the Python DLL #else #define PyAPI_DATA(RTYPE,NAME) // PyAPI_DATA can be blank since then the real declarations come from dataexports.h #endif pydebug.h and all other files that declare exported data: PyAPI_DATA(int) Py_VerboseFlag; --> PyAPI_DATA(int, Py_VerboseFlag) In addition, complex types such as function pointers and arrays would need to be typedef'ed, e.g.: PyAPI_DATA(char) *(*PyOS_ReadlineFunctionPointer)(FILE *, FILE *, char *); --> typedef char *(*t_PyOS_ReadlineFunctionPointer)(FILE *, FILE *, char *); PyAPI_DATA(t_PyOS_ReadlineFunctionPointer, PyOS_ReadlineFunctionPointer) and multiple PyAPI_DATA declarations on a line would need to be split to several PyAPI_DATA declarations. If this change isn't acceptable then we'd pretty much have to fork this aspect of the interpreter - either by just maintaining a patch that's applied to the headers or by making a preprocessor that fixes the headers on the fly. Does this sound reasonable? Any better ideas? -Jukka

2 1

Re: [Python-Dev] Coverity Scan, Python upgraded to rung 2
by Christian Heimes 09 Jan '08

09 Jan '08

Joseph Armbruster wrote: > Christian, > > Is there any way you (or someone else) could post up the results? It > looks like you need a log in to check them out. I haven't figured out how to access the results. Who has a login and access to the site? Christian

2 1

Import by filename with __import__ ?
by Christian Heimes 08 Jan '08

08 Jan '08

Today I stumbled about an unknown and undocumented (?) feature. At least it's not documented in our docs. __import__ can import a module by file name: >>> open("/tmp/example.py", "w").write("test = 23\n") >>> mod = __import__("/tmp/example") >>> mod <module '/tmp/example' from '/tmp/example.py'> >>> mod.__name__ '/tmp/example' >>> mod.__file__ '/tmp/example.py' >>> mod.test 23 Is it just a coincidence? Is it a desired feature? Why isn't it documented? Christian

2 1

Extracting variables from string.Template objects
by Isaac Morland 08 Jan '08

08 Jan '08

I found myself wanting to obtain the variables from a string.Template object. The situation is that part of input consists of a filename template, and I want to make sure that all the parameter values I have are actually represented in the filename template. So: def templateVariables (template): """Extract the variable names from a string.Template. Returns a tuple of all variable names found in the template, in the order in which they occur. If an invalid escape sequence occurs, the same error will be raised as if an attempt was made to expand the template. """ result = [] for match in template.pattern.finditer (template.template): if match.group ('invalid') is not None: # Raises ValueError template._invalid (match) if match.group ('escaped') is not None: continue # The "or None" should be moot. It is there to ensure equivalent # treatment for an empty 'named' and an empty 'braced'. result.append (match.group ('named') or match.group ('braced') or None) return tuple (result) Note that almost all the work is actually done by calling in to parts of the template object, so I think this should work properly with subclasses and other less common cases. I would like to add this as a method of string.Template, which I think amounts to changing "template" to "self" and putting it in the Template class in string.py rather than on its own. If the general idea is approved I would be happy to make a patch. Also, on a related issue, does it make sense to scan the template string for invalid escape sequences in Template.__init__? For the applications I can imagine of string.Template, I would prefer to get an error upon creating the Template object rather than arbitrarily later when I try to .substitute with it. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist

2 3

Backport PEP 3129: Class Decorators
by Raymond Hettinger 08 Jan '08

08 Jan '08

This seems like something that could reasonably be added to Py2.6. Raymond

4 6

test_sys failures
by Guido van Rossum 08 Jan '08

08 Jan '08

When I build from scratch and run most tests (regrtest.py -uall) I get some strange failures with test_sys.py: test test_sys failed -- Traceback (most recent call last): File "/usr/local/google/home/guido/python/py3kd/Lib/test/test_sys.py", line 302, in test_43581 self.assertEqual(sys.__stdout__.encoding, sys.__stderr__.encoding) AssertionError: 'ascii' != 'ISO-8859-1' The same test doesn't fail when run in isolation. Interestingly, I saw this with 2.5 as well as 3.0, but not with 2.6! Any ideas? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

4 5

pkgutil, pkg_resource and Python 3.0 name space packages
by Raymond Hettinger 07 Jan '08

07 Jan '08

>> And then extend this to any other >> package that we consider creating? Otherwise leave it out? How would >> that follow for sqlite since that is not going to get any shorter >> thanks to a package? Should it still go into the package for >> organizational purposes? > If you're asking me, the "organizational purpose" is 100% misguided. It is my hope that there will be a great deal of restraint in the effort to group modules into packages in Py3.0. For the most part, putting modules in packages only makes them more difficult to access and introduces an additional burden of remembering which package to invoke. The best existing indicator we have is the organization of the docs for the standard library. I, for one, have a hell of a difficult time finding modules via the "organized" table of contents in the Library Reference. Instead, I always go the the Global Module Index where the somewhat flat namespace makes it easy to go directly to the module of interest. I'm curious whether the other developers have had the same experience -- if so, then it is a bad omen for over-organizing the standard library. Another indicator of what lies ahead is the current organization of os vs os.path. While that split-out was well done and necessary, I routinely have difficulty remembering which one contains a function of interest. There are handful of groupings that are obvious (i.e. html and socket modules going into an internet package). Outside of those, I think it best to leave the rest of the modules in a flat namespace (I don't want to have to quess where to find queue, random, pickle, pprint, decimal, and collections. I think most of the concerns with the standard library revolve around module quality, duplication, and obsolence. In contrast, I do not think that introducing a hierarchy of namespaces is of interest to most users. That exercise may well make the language harder to use/learn, not easier. my-two-cents-ly, Raymond

5 4

Re: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages
by Phillip J. Eby 07 Jan '08

07 Jan '08

At 04:23 PM 1/6/2008 -0800, Guido van Rossum wrote: >Regarding using common words, either the stdlib grabs these, or >*nobody* gets to use them (for fear of conflicting with some other 3rd >party package grabbing the same). This isn't quite true; a standalone Python application that isn't extensible doesn't need to worry about this. And it's standalone apps that are most likely to claim these common words. (For example, until recently, Chandler's database library packages were in 'repository.*'.) But of course this is still a pretty minor point overall. If the stdlib does go for deeper nestings, I have a slight preference for seeing them under std.* or some such rather than top level. But I don't really see a whole lot of point to doing a major re-org of the stdlib space to begin with, for the simple reason that package names are not really categories -- they're *names*. IMO 'from databases import sqlite' doesn't add any value over 'import pysqlite3' to begin with. Worse, it will likely be an attractive nuisance for people saying "why don't we have databases.Oracle?" and suchlike. And you still have to remember the names, only now they're longer. And was it database or databases? Greater uniqueness of names is just another reason flat is better than nested. :)

4 7

Do we need both Trac and Roundup?
by skip＠pobox.com 07 Jan '08

07 Jan '08

A ticket I opened on Roundup about a website issue some time ago was closed today. It was related to a different topic, but in the discussion I wrote: Is there some reason at this point that we need to maintain two separate trackers? A ton of work went into making our Roundup instance what the key players wanted. Why not use it to track website issues as well? With them separate you have two communities of maintainers and triage folks who are mostly disjoint. I assume there would be some trac-to-roundup conversion necessary. Perhaps something like that already exists. Do people think it would be worthwhile to merge the Trac tracker content (the issue tracker that holds tickets related to the python.org website) into the Roundup tracker (the issue tracker that holds tickets related to Python the language)? While they are nominally distinct activities, it seems to me that we must waste some precious resources (mostly people) maintaining two mostly disjoint trackers. There is also some functionality overlap, mostly in the documentation area. Skip

4 3

Re: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages
by Phillip J. Eby 07 Jan '08

07 Jan '08

At 02:10 PM 1/6/2008 -0800, Brett Cannon wrote: >My question becomes whether we want to allow something like this even >if we explicitly state people should not use this mechanism to >override pre-existing modules. Do we want people tossing stuff into >the 'databases' package, or should the packages in the stdlib be >considered sacred? This is actually an excellent point, given that the actual intended use of namespace packages is to allow an *organization* to control a namespace: e.g. zope.* and zc.* packages, osaf.* packages, etc. Using names that have meaning (like "email" or "databases") sort of goes against the whole point of namespace packages in the first place. For some reason, I wasn't thinking about that when the original post came through. So, now that I've thought about it, I'm -1 on the stdlib using *top-level* namespace packages to sort out its contents (e.g. "databases" as a top-level package name) If we want to allow separately-distributed *upgrades* or bugfix releases of projects (such as an updated sqlite module), then using 2nd-level namespace packages like "std.databases.*" would allow that. Note, by the way, that this implies that somebody creating their own Oracle driver would *not* be expected to put it into std.databases. Again, the whole point of a namespace package is to reserve that namespace for packages produced by a particular organization, similar to the way e.g. the 'org.apache.projectname' packages in Java work.

4 3