On Wed, 21 Apr 1999, Guido van Rossum wrote:

...

Guido, how do you envision python-dev working?

I though we'd just crack Monty Python jokes while pretending to work :-)

That'd be better than what I've read on the perl5-porters list in the last couple of days (which I'm reading because of the ad hoopla). They seem to spend their time arguing about whether Perl codes are programs or scripts. Viciously, too. =) More seriously, some of the discussions they're having a worth keeping an eye on, just as a form of legal industrial spying.

Import revamp

There was an interesting bit from Greg, Jack, and MarkH on this topic recently in the distutil-sig. Maybe one of them can bring the rest of us up to date on the effort. --david

[Guido and /F]

...

Unicode

What is the status of /F's latest attempts?

most of the code is written and tested; the API still needs some cleaning up. I also have a few patches and other contributions in my inbox that I gotta look into...

Im one of those contributors :-) I have made changes to the Win32 extensions that allows it to use /F's type instead of its own built-in Unicode type. Further, the Windows CE port of these extensions actually enables this - so we have some code actively using it. However, this still doesnt give me anything more than I had before. We need at least _some_ level of integration of this type into Python. The key areas to me are: * PyArg_ParseTuple and Py_BuildValue - Seems OK we use the "u" format character analogous to "s". We havent thought of a "z" equivilent. This is fairly easy. * Some way for these functions to auto-convert. Eg, when the user passes an 8bit string object, but the C side of the world wants a Unicode string. This is harder, as an extra memory buffer for the conversion is needed. This second problem is, to me, the next stumbling block. When we sort these out, I feel we would have the start of a good platform to start experimenting. Mark.

David Beazley wrote:

Guido van Rossum writes:

...

ANSI C

I'm tired of supporting K&R C. All code should be converted to using prototypes. The Py_PROTO macros should go. We can add const-correctness to all code. Too bad for platforms without decent compilers (let them get GCC).

Maybe I'm just lucky, but I've written nothing but ANSI C for the last 8 years and have never had a problem compiling it on any machine. I don't see this as being a huge issue.

I've had the same experience with things like prototypes. The recent introduction of indented preprocessor instructions caused me significant pain on some platforms. Fortunately, I haven't had to deal with those platforms recently, still, the elegence of indented preporicessor instructions wasn't worth the pain it caused. As far as choice of compilers goes, sometimes gcc isn't an option. I've had cases where I *had* to use the native compiler in order to link to third-party libraries. I support moving to ANSI C, for example, wrt prototypes, but lets still be somewhat sensative to portability and not abuse bad compilers without good reason.

...

Misc

Depending on how deep I want to dig, I may have some patches to ParseTuple() that would make life easier for some of us extension writing fans. As it is right now, my experimental version of Swig is using it's own version of ParseTuple (for various reasons). Also, out of curiousity, is anyone making heavy use of the CObject type right now?

Yes. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

David Beazley wrote:

...

Misc

Depending on how deep I want to dig, I may have some patches to ParseTuple() that would make life easier for some of us extension writing fans. As it is right now, my experimental version of Swig is using it's own version of ParseTuple (for various reasons). Also, out of curiousity, is anyone making heavy use of the CObject type right now?

You mean the one in Objects/cobject.c ? Sure, all my type extensions export their C API that way... and it works just great (no more linker problems, nice error messages when extensions are not found, etc.). -- Marc-Andre Lemburg Y2000: 254 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : ---------------------------------------------------------

Guido van Rossum wrote:

Break up into VM, parser, import, mainloop, and other components?

The porting efforts to seriously small machines (Win/CE, Pilot, etc.) all seem to need a Python VM that doesn't automatically pull in the parser and interactive main loop. There are other reasons as well to isolate various parts of the functionality (including all numeric data types except ints).

This could include: restructuring of the parser so codeop.py can be simplified; making the interactive main loop a script.

There's some good work out there by Skip Montanaro to revamp the VM into a combined register/stack machine. He calls it rattlesnake. More infos can be found in the list archive (it's currently sleeping): http://www.egroups.com/folders/rattlesnake I would like to see the VM/compiler pluggable so that experiments like rattlesnake become easier to implement (extension modules vs. patches to the core interpreter).

Coercions

Marc-Andre Lemburg has some good suggestions here: http://starship.python.net/~lemburg/coercion-0.6.zip

I will continue to work on these patches now that 1.5.2 is out. There is a (more or less) detailed explanation of the patch set at: http://starship.skyport.net/~lemburg/CoercionProposal.html Instead of using of using the PY_NEWSTYLENUMBER approach I'll turn to the new type flags that were introduced in 1.5.2. The Py_NotImplemented singleton return value will stay, because performance tests have shown that this method does not cause any significant hit where as the exception raise and catch method does introduce a significant slow-down.

Import revamp

Greg's imputil.py (it's in the distutils package) should provide a good start.

Buffer object API extensions

I'll put the complete patch up on starship later this week... here are the basic prototypes: DL_IMPORT(int) PyObject_AsCharBuffer Py_PROTO((PyObject *obj, const char **buffer, int *buffer_len)); /* Takes an arbitrary object which must support the (character, single segment) buffer interface and returns a pointer to a read-only memory location useable as character based input for subsequent processing. buffer and buffer_len are only set in case no error occurrs. Otherwise, -1 is returned and an exception set. */ DL_IMPORT(int) PyObject_AsReadBuffer Py_PROTO((PyObject *obj, const void **buffer, int *buffer_len)); /* Same as PyObject_AsCharBuffer() except that this API expects (readable, single segment) buffer interface and returns a pointer to a read-only memory location which can contain arbitrary data. buffer and buffer_len are only set in case no error occurrs. Otherwise, -1 is returned and an exception set. */ DL_IMPORT(int) PyObject_AsWriteBuffer Py_PROTO((PyObject *obj, void **buffer, int *buffer_len)); /* Takes an arbitrary object which must support the (writeable, single segment) buffer interface and returns a pointer to a writeable memory location in buffer of size buffer_len. buffer and buffer_len are only set in case no error occurrs. Otherwise, -1 is returned and an exception set. */

Distutil

Should we start integrating some of the results of the distutil SIG? (What's the status? I haven't been paying attention.)

With all the different modules and packages available for Python starting to slowly introduce dependencies, I think the main objective should be introducing a well organized installation info system, e.g. a registry where modules and packages can query and save version and dependency information. Someone on the main list mentioned that JPython has something like this...

Misc

Jeremy Hylton's urllib2.py

Greg Stein's new httplib.py (or Jeremy's?)

Andrew Kuchling's new PCRE release?

The IPv6 patches?

These are on my wish list, so I'll simply add them here: · APIs to faciliate creation of extensions classes (using Python classes and C functions as methods) The main open question here is where and how to store C data in the instances. This could also provide a way to migrate to all Python classes for some future version. · Restructure the calling mechanism used in ceval.c to make it clearer, more flexible and faster (of course, :-). Also, inline calling of C functions/methods (this produces a noticable speedup). I have a patch for the restructuring and an old one (against 1.5) for the C function call inlining. · A "fastpath" hook made available through the sys module. This should hook into the module/package loader and should be used whenever set to find a module (reverting to the standard lookup method in case it can't find it). I have an old patch that does this. It uses a simple Python function to redirect the lookup to a precalculated dictionary of installed modules/packages. This results in faster startup of the Python interpreter (by saving a few hundred stat() calls). -- Marc-Andre Lemburg Y2000: 254 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : ---------------------------------------------------------

Guido van Rossum wrote:

... Import revamp

It should be much easier to add hooks to allow importing from zip files, from a URL, etc., or (different) hooks to allow looking for different extensions (e.g. to automatically generate ILU stubs). This should help the small platforms too, since they often don't have a filesystem, so their imports will have to come from some other place.

I would recommend that Python's import mechanism be changed to use some of the concepts from my imputil.py module and in a couple of my posts to distutils-sig: http://www.python.org/pipermail/distutils-sig/1999-January/000142.html http://www.python.org/pipermail/distutils-sig/1998-December/000077.html

ANSI C

I'm tired of supporting K&R C. All code should be converted to using prototypes. The Py_PROTO macros should go. We can add const-correctness to all code. Too bad for platforms without decent compilers (let them get GCC).

Or those platforms can stick with Python 1.5.2

Misc

Jeremy Hylton's urllib2.py

Greg Stein's new httplib.py (or Jeremy's?)

Mine is a drop-in replacement for the current httplib.py. httplib.HTTP is compatible with the current version and does HTTP/1.0. httplib.HTTPConnection is similar to the other (but (IMO) cleans up a few elements of the API); it does HTTP/1.1, including persistent connections. This module follows the design of the original httplib.py much more closely. Jeremy's module uses a very different design approach (no value judgement here; it's just different); also, it simply copies over the old HTTP class, while mine derives the compatible version from HTTPConnection. I'm not sure how much testing Jeremy's has had, but I use my module for all Python-based http work now. Some number of other people have downloaded and used it, too. I would recommend my new module, given the two options. I'll eventually have a davlib.py to add to the distribution, but it needs more work. I'll contribute that later. --- I'd also like to see some cleanup of the build process. For example, (Guido:) I sent in a different way to handle the signal stuff (shifting files around and stuff) as opposed to the "hassignal" thing. About the build process: it may be nice to move to a single makefile. Here is an interesting paper about this: http://www.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html Is it reasonable to split out the Demo, Doc, Misc, and Tools subdirs since those never really end up as part of an installed Python? e.g. into a python-extras distribution? (which would allow doc and tools updates at arbitrary times) Does it make sense to remove platform-specific modules? (e.g. the SGI modules) While distutils doesn't have anything *yet*, there are changes that can be made in the build process to improve the distutils process. I know that Greg Ward wants distutils to work on previous versions of Python, but (IMO) things would be simpler all around if distutils-based packages were simply designed for 1.6 systems as a basic dependency. However, that choice is quite separate from the decision to generate distutils support during the build. Oh: there were some things floating around at one point about using Python itself within the build process (e.g. for modules). Is any of that work useful? Cheers, -g -- Greg Stein, http://www.lyra.org/

Fred> I certainly think that doing something similar for the Demo/ and Fred> Tools/ directories would be reasonable; they could share a single Fred> distribution. We need to make sure none of the tools are needed in the install process. h2py comes to mind. Skip Montanaro | Mojam: "Uniting the World of Music" http://www.mojam.com/ skip@mojam.com | Musi-Cal: http://www.musi-cal.com/ 518-372-5583