Mailman 3 May 2004 - Python-Dev

ANN: Job Openings at Enthought
by eric jones 17 May '04

17 May '04

Hello, This is a bit of a status report I guess for SciPy (www.scipy.org) and Enthought (www.enthought.com), but, really, it is a long winded job posting. Among other things, Enthought develops scientific software for clients. Python is central to to our strategy in this arena. We have long supported SciPy and believe strongly in Open Source Software. The purpose is to give people a feeling about what we do, the technical challenges we face, and the skills needed to meet them. SciPy ----- There has been a lot of work on SciPy lately by the Travis Oliphant, Travis Vaught, Pearu Peterson, Joe Cooper, and on the website by Jon-Eric Steinbomer and Janet Swisher. The site is now upgraded to Plone and we have a SciPy 0.3 package out. If you're interested in seeing the activity level at scipy.org, please visit. http://www.scipy.org/map?rmurl=http%3A//scipy.net/scipyusage/ It looks like we're averaging about a 100 downloads a day if you add up all source and binary distributions. When SciPy 0.1 was released a couple a years ago, I think we averaged about 10. Its extremely difficult to extrapolate a user base from such information, the obvious growth is good news. Other Stuff ----------- In addition to SciPy we have multiple projects going on right now that either already plug into or will be plugins to a Python-based scientific application framework called Envisage (sorta like an IDE for science apps) that we're working on. The idea is: (1) To lower the bar as far as work required to get a GUI put on the front end of a scientific algorithm. (2) Develop scientific plugins that can play well together so that, for instance, an electromagnetics simulator built by one person can be used in conjunction with an optimization plugin built by another person and results can be visualized with a 3D plugin. This is a hard, long path that requires much work and though. We have started the process and actually have built an app on a very early version of the Envisage framework. The app works great, but it's usage of the framework is a little mixed at this point. There were a lot of compromises we ended up making on the framework side in order to ship on time. Also, I am not sure whether "easy-to-program" and "flexible architecture for compatible plugins" are not mutually exclusive. I always have in my mind that I want a smart scientist to be able to build the GUI in a short amount of time after the algorithm (which is the "hard part") is finished. I still harbor this wish, but the lessons I've had over the last couple of years suggest that the GUI is actually the most expensive part of development for large apps. This is partially due to the fact that commercial apps are rarely built around "research" algorithms -- meaning that the algorithm code usually already exists in some form. Still, UIs take tons of time. Testing them is hard. Flexibility requires more complexity (factories, adapters, etc.) than seems necessary at first. Further, building *good* UI's is really hard. Scientist rarely have the patience or perspective for it -- and yet it is very often difficult to build a good UI for science apps without a scientist's understanding of the underlying problem. We have a number of tools along with SciPy that we're building that are pieces of this puzzle. 1. Traits -- They at the heart of everything we do. They provide some explicit typing, default GUI's, an observer event model for anything that uses them. They can also require some getting used to. 2. Kiva/Enable -- This is the generic drawing system for Python. 3. Chaco -- Interactive 2D Plotting 4. PyFace -- MVC layer on top of wxPython. Supports trees, menus, and a few other things right now. This will grow. 5. Envisage -- Plugin based GUI framework for scientific applications. Beyond the basic GUI that comes from Envisage, we already have a couple plugins for profiling code using hotshot and also searching for memory leaks. David Morrill wrote these and they are called gotcha and enroll. Martin Chilvers wrote a simple scintilla based plugin for editing scripts, and a PyCrust shell is available by default in the app (not a plugin at the moment). We would love to see a number of other plugins: a. Debugger. b. IPython-like PyCrust going. c. A more full featured script editing plugin. (there is a huge list of features here). d. Mayavi e. etc. These are all used actively in commercial products that we deliver. However, they are also in various stages of development and will continue to evolve and grow. Some are still pretty green. Portions of these are openly available now. All five of the listed tool sets will be open at some point this summer when they are cleaned up a bit. Jobs ---- All of this leads to the fact that we are hiring and looking for smart, pleasant, communicative people with integrity that are excited about building this sort of stuff. There is a lot of software architecture to be done that needs talented designers. There is UI design/development that needs scientists that care about UIs or UI developers/designers that are really quick at grasping scientific problems. We also need really strong scientists/scientific developers to do some of the backend coding for commercial products and also SciPy development. If you have a background in geophysics, electromagnetics (especially multi-pole methods) that is a big plus. For this, a PhD is helpful, but not required. Parallel computing skills wouldn't hurt. 3D visualization a la VTK/Python is a major need. So is knowledge about 2D rendering to help finish out Kiva and the Enable toolset. Strong Python skills, Python extension writing knowledge and strong C/C++ skills are a big benefit. Design/development with or of Java NetBeans or Eclipse-like architecture is great -- or any other solid GUI architecture. Dedication to writing clean/clear code meant for other humans to read is a requirement. We have 3 or 4 scientific/python positions to fill over the coming months, and we're looking for candidates that have the best mix of the above skills. You'll join our existing team of scientist/engineers, computer scientists, technical writers, and an HCI specialist. We like this mix and feel it is the best way to build this sort of software. If you are interested in working at Enthought, please send us your resume at jobs(a)enthought.com. If not, please send this posting to the smartest people you know that fit some part of the above description (Python experience not explicitly required). Salaries are competitive. Candidates must be willing to relocate to Austin, Tx. thanks, eric jones PS: There are additional positions listed at http://www.enthought.com/careers.htm for those interested in business application development (not necessarily with Python).

1 0

RE: [Python-Dev] .pth files in current or script directory
by Delaney, Timothy C (Timothy) 17 May '04

17 May '04

Guido van Rossum wrote: > Can't they manipulate sys.path *inside* the code in question? That's > invariably the solution I end up using -- I never play with > PYTHONPATH any more. I do this too, but I always feel that I shouldn't have to, especially when I'm not using `sys` for anything else. It also messes up my desire to keep all the imports together - the `sys` magic needs to go first. Jeff Epler wrote: > I'm -NFS against reading .pth files from the current directory. > When a directory is large and is on a remote filesystem, enumerating > it at startup can really kill time -- heck, just putting an NFS > directory with a handful of entries in PYTHONPATH can slow Python's > startup time considerably, now imagine that you "cd > /nfs/directory_with_1000_files" and "time python -c 'pass'" .. blah! I don't use NFS so I've no experience with the performance problems involved. Of course, Windows shares would have similar issues. Hmm - would this objection go away if a specific filename were mandated for these .pth files - e.g. pythonpath.pth? Tim Delaney

3 2

RE: [Python-Dev] .pth files in current or script directory
by Delaney, Timothy C (Timothy) 17 May '04

17 May '04

Jeff Epler wrote: > On Mon, May 17, 2004 at 10:11:23AM +1000, Delaney, Timothy C > (Timothy) wrote: >> I don't use NFS so I've no experience with the performance problems >> involved. Of course, Windows shares would have similar issues. >> >> Hmm - would this objection go away if a specific filename were >> mandated for these .pth files - e.g. pythonpath.pth? > > Yes, searching for a single named file is a much faster operation than > listing a directory, and would probably be a fairly small impact on > Python startup, much smaller than listing a directory with many files. > It's one round-trip to the server vs many round trips. > > .. but that makes the feature somewhat different than .pth files in > site-packages (any name vs a single name), which may be a bad thing > too. Yep - I thought about this. OTOH, the use case for .pth files for site-packages is multiple libraries packaged separately. The use case for this feature is a single code base i.e. there would be no need for multiple .pth files since everything is under control of the developer. Please don't get the idea I'm all gung-ho about this - I'm just trying to see if there's enough reason to write a PEP for it (either to be accepted or rejected). Tim Delaney

1 0

Re: [Python-Dev] Berkeley test errors in 2.3.4c1
by Gregory P. Smith 17 May '04

17 May '04

On Sun, May 16, 2004 at 01:47:03AM -0400, Tim Peters wrote: > [Gregory P. Smith, on test_bsddb3 Win98SE failures in test01_basics] > > Yeah I'd chalk that one up to Win98. > > Me too <wink>. > > Back over on WinXP and *current* (HEAD) CVS, there's one failure, which I > believe is real: > > ERROR: test04_n_flag (bsddb.test.test_compat.CompatibilityTestCase) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Code\python\lib\bsddb\test\test_compat.py", line 86, in > test04_n_flag > f = hashopen(self.filename, 'n') > File "C:\Code\python\lib\bsddb\__init__.py", line 293, in hashopen > d.open(file, db.DB_HASH, flags, mode) > DBInvalidArgError: (22, 'Invalid argument -- DB_TRUNCATE illegal with > locking specified') > > > I suspect Sleepycat screwed us there, changing the rules in midstream. > Someone on c.l.py appeared to track down the same thing here, but in an app > instead of in our test suite: > > http://mail.python.org/pipermail/python-list/2004-May/220168.html > > The change log of Berkeley DB 4.2.52 says "9. Fix a bug to now > disallow DB_TRUNCATE on opens in locking environments, since we > cannot prevent race conditions ..." heh. nice. yeah i saw that bugreport for pybsddb but hadn't looked into it yet. Binary packages of python 2.3.4 should be compiled using BerkeleyDB 4.1 not 4.2 (as all previous python 2.3s shipped with 4.1). Anyone who compiles it on their own against 4.2 will have to live with this error in the old bsddb compatibility interface. Taking a closer look at this to see if there is a workaround is on my TODO list. -g

1 0

RE: [Python-checkins] python/dist/src/Lib/email Parser.py, 1.20, 1.20.12.1
by Tim Peters 16 May '04

16 May '04

[Barry] > Log Message: > _parsebody(): Do not create subparts unless the container has a main > type of 'multipart' and the boundary is defined. This fixes SF bug # > 846938, and several recent email-sig bugs ... Yadda yadda yadda. At this point, the more obvious solution is to ignore all that MIME crap, and just guess a good structure for the email. The spambayes Outlook addin does that all the time <wink>.

5 5

Berkeley test errors in 2.3.4c1
by Tim Peters 16 May '04

16 May '04

I tried 2.3.4rc1 on Win XP SP1, and got 30 errors in test_bsddb3. Opened a bug report: http://www.python.org/sf/954530 How's test_bsddb3 working on other 2.3.4c1 installations (I encourage adding info to the bug report instead of via email)?

2 4

Re: Welcome to the "Python-Dev" mailing list
by Michel Van den Bergh 16 May '04

16 May '04

Hello, I apologize for posting so quickly on this list but I am having a problem. I am the author of a game written in python (PyTraffic, it is on the vault) and I am preparing a new version which will be much nicer in many ways. It also includes some help in the form of a html file. Right now I am using my own methods to display html files. However I am also checking out the webbrowser module. Unfortunately this module has some obvious problems (tested on CVS version under Linux with mozilla) (*) If you open an url (with the option "new window") and the browser is not already running then it opens *two* windows: the home page and your url. This is different for example from the default behaviour on Windows and also quite ugly. (**) There is another related problem: if you open an url and the browser is not already running then the code starts a new browser, does a timed sleep and then issues a "-remote" command. Using a timed sleep is very bad programming and *extremely fragile*. Indeed I had failures because the delay (PROCESS_CREATION_DELAY) was not long enough. At the very least the -remote command should be tried a couple of times. BUT: there is no reason *at all* to issue a -remote command in this case. All browsers I know accept a url as argument. So one can just start a new browser with the url as argument. These problems are easy to fix but it is not clear from which version of webbrowser.py to start. There are already two patches waiting (not fixing the above problems), with no indication if they will ever be applied. Any comments? Best regards, Michel

4 3

SF CVS question
by Thomas Heller 15 May '04

15 May '04

When doing 'cvs -n update', I get a warning like this: cvs server: WARNING: global `-l´ option ignored. Does anyone else get the same? Is there something wrong with my configuration? It also occurrs on other SF cvs repositories, not only with python. Thomas

2 3

modifying input behavior with PyOS_ReadlineFunctionPointer
by Gregory Lielens 15 May '04

15 May '04

Hello all, I have posted this to the general comp.lang.python list, But as it is really about an implementation detail, it is probably more a topic for the development list I apologize in advance if this is not the correct place to post this.... We are currently developing python bindings to make a MPI parallel application (Finite element solver) script driven. I have written a main interpreter, able to execute scripts or interactive commands, in // , based on the following techniques: *file execution: all python interpreter processes run the file using PyRun_SimpleFile (file must be available to all processes) *file execution using broadcast: process 0 read input file and store it in a file, broadcast it to all processes and the script is run using PyRun_SimpleString (file must be available to process 0 only) *interactive execution: All processes listen to user input using a PyRun_InteractiveLoop. Process 0 read stdin and broadcast the line to all process, the other process listen to this broadcast. This is implemented using the (very useful in this case :-) ) PyOS_ReadlineFunctionPointer , reassigning it to an input function which wrap the default PyOS_ReadlineFunctionPointer (to be able to reuse nice readline functionalities) and add the broadcasting mechanism... All of this work perfectly, but I had to modify the python sources, Parser/myreadline.c to be precise... Indeed, the behavior of PyOS_Readline is to use the user-modifiable PyOS_ReadlineFunctionPointer as input mechanism, except when input is non-interactive in which case it fall back to the non-user-modifiable PyOS_StdioReadline...As my processes are non-interactive except for proc 0 (launched in background), proc 0 broadcast but the other ones do not listen. I thus had to remove this test, so that the user-modifiable function is called in all cases. /*Python code snipped from Parser/myreadline.c*/ char * PyOS_Readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt) { char *rv; if (PyOS_ReadlineFunctionPointer == NULL) { #ifdef __VMS PyOS_ReadlineFunctionPointer = vms__StdioReadline; #else PyOS_ReadlineFunctionPointer = PyOS_StdioReadline; #endif } } Py_BEGIN_ALLOW_THREADS /* This is needed to handle the unlikely case that the * interpreter is in interactive mode *and* stdin/out are not * a tty. This can happen, for example if python is run like * this: python -i < test1.py */ /* my modif: comment out the use of PyOS_StdioReadline for non-interactive input...*/ /*if (!isatty (fileno (sys_stdin)) || !isatty (fileno(sys_stdout))) rv = PyOS_StdioReadline (sys_stdin, sys_stdout, prompt); else*/ rv = (*PyOS_ReadlineFunctionPointer)(sys_stdin, sys_stdout, prompt); Py_END_ALLOW_THREADS return rv; } } /*end of snipped code*/ This is a small modif, but annoying because we would prefer to use the plain python distribution, and anyway I wonder if the current behavior of python is desirable, because basically it removes part of the functionality of PyOS_ReadlineFunctionPointer: flexibility disappear for non-interactive input. In addition, this seems not so robust, it seems to me that it is not correct for vms, where PyOS_StdioReadline will be used in non-interactive case while vms__StdioReadline will be used for interactive one....maybe this is the intended behavior, but then the function naming is strange :-) So basically, shouldn't it be the responsibility of the code which change PyOS_ReadlineFunctionPointer to check if the function given is adapted to interactiveness of input? this seems the more flexible approach, and probably involve only minor modif of the readline module. In addition, wouldn't it be nice to initialize PyOS_ReadlineFunctionPointer to a default value (suitable reading function) at declaration, instead of defining it to NULL and let PyOS_Readline do the initialization when needed? This way, user code can get back a meaningful reading function storing the old value of PyOS_ReadlineFunctionPointer, use it to write an extended input function, and reassign it to PyOS_ReadlineFunctionPointer... This seems to me like the most flexible way to add user-tunable input method... So, to all experienced python developers, having used this PyOS_ReadlineFunctionPointer function or implemented this code, what do you think of these possible slight modifications? Or is there a better way to use it than my current method? Sorry for the long (and possibly strangely written, English is not my native language :-) ) post, Best regards, Greg.

1 0

RE: [Python-Dev] Is core dump always a bug? Advice requested
by Michel Pelletier 15 May '04

15 May '04

On Thursday 13 May 2004 04:43, python-dev-request(a)python.org wrote: > It's important to read what people who have actually done it have to say > about subtleties. For example, your algorithm considered 'return' to be 'a > terminal instruction'. But what if it appeared in the 'try' clause of a > try/finally construct? In Python's PVM, most *possible* control flow is > implicit (virtually any opcode can raise an exception, and from there > "magically jump" to the code at an enclosing 'except' or 'finally' clause); > and even an unexceptional bare 'return' can magically jump to an enclosing > 'finally' block. The Java reference discusses these things, and anyone > intending to do something here is still strongly urged to read it. But just looking at some code here, is RETURN_VALUE ever inside a SETUP_EXCEPT block? The end of the SETUP_EXCEPT block apears to ABSOLUTE_JUMP past all the exception handlers to finally, and then to the RETURN_VALUE, which doesn't seem to fall under the protection of a try/except. I'm not saying there are no subtleties as you describe, I'm just trying to quantify some of them and wondering if this is one. -Michel

3 3