Mailman 3 January 2004 - Python-Dev

Re:[Python-Dev] collections module (correction)
by Christos Georgiou Jan. 10, 2004

Jan. 10, 2004

And a correction to the size method I suggested for the queue object: def size(self): return sum(map(len, self.superlist)) The previous one was giving wrong results when get was called N times and N % maximum_sublist_size != 0. I believe this should be implemented in C, otherwise the overhead would be too much for queues that never grew.

1 0

Fixes for imageop module
by Sjoerd Mullender Jan. 10, 2004

Jan. 10, 2004

I have some fixes for the imageop module to make it work on systems with a different byte-order than IRIX (e.g. Linux). Shall I check them in or is it not worth it for such an old module? Anything else that needs to be changed when I check this in? The fixes of course will result in different results on Linux than before, so that's the main reason for asking. -- Sjoerd Mullender <sjoerd(a)acm.org>

2 9

trying to build 2.3.3 with db-4.2.52
by Matthias Klose Jan. 9, 2004

Jan. 9, 2004

Trying to build python2.3.3 with db-4.2.52, I get many failures in various tests: 5 tests failed: test_anydbm test_bsddb test_dircache test_shelve test_whichdb all of the same type: test test_whichdb failed -- Traceback (most recent call last): File "/home/packages/python/2.3/python2.3-2.3.3/Lib/test/test_whichdb.py", line 49, in test_whichdb_name f = mod.open(_fname, 'c') File "/home/packages/python/2.3/python2.3-2.3.3/Lib/dbhash.py", line 16, in open return bsddb.hashopen(file, flag, mode) File "/home/packages/python/2.3/python2.3-2.3.3/Lib/bsddb/__init__.py", line 192, in hashopen d.open(file, db.DB_HASH, flags, mode) DBError: (38, 'Function not implemented -- process-private: unable to initialize environment lock: Function not implemented') using db-4.1.25, all the tests succeed. any hints? The build environment is a current Debian GNU/Linux unstable i386 system. Matthias

3 2

Re: [Python-Dev] The os module, unix and win32
by Barry Scott Jan. 9, 2004

Jan. 9, 2004

I've got code that implements the popen5 like functionality in my Emacs do you to look it over? Its based on sample code from microsoft. In http://www.barrys-emacs.org/bemacs_7.2-230.win32_proc_code.zip you will find the C++ modules. I can guess from your API list that what you have will probably not work from pythonw, I recall that you need to call AllocConsole for example have play games with std handles. Barry At 08-01-2004 22:31, Peter Astrand wrote: > > > win32all covers a huge number of API functions, more then would > > > be sane to add to os. But would there be any mileage in added > > > enough from win32all to allow problems like popen5 to be > > > implemented? > > > > > > There is already the _reg module that has some win32 functions in > > > it on the standard install. > > > > (Not sure if this was in response to my "let's do it in Python" post.) > > > > How many APIs would have to be copied from win32all to enable > > implementing popen5 in Python? > >I'm not sure since I haven't implemented much yet, but we'll need at >least: > >win32.CreatePipe >win32api.DuplicateHandle >win32api.GetCurrentProcess >win32process.CreateProcess >win32process.STARTUPINFO >win32gui.EnumThreadWindows >win32event.WaitForSingleObject >win32process.TerminateProcess > > >-- >/Peter Åstrand <astrand(a)lysator.liu.se> > > > > >_______________________________________________ >Python-Dev mailing list >Python-Dev(a)python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/nospam%40barrys-emacs.org

1 0

Re: [Python-Dev] The os module, unix and win32
by Barry Scott Jan. 9, 2004

Jan. 9, 2004

For pythonw execution you will need AllocConsole and SetStdHandle to allow CMD.EXE to be run. At 08-01-2004 22:31, Peter Astrand wrote: >I'm not sure since I haven't implemented much yet, but we'll need at >least: > >win32.CreatePipe >win32api.DuplicateHandle >win32api.GetCurrentProcess >win32process.CreateProcess >win32process.STARTUPINFO >win32gui.EnumThreadWindows >win32event.WaitForSingleObject >win32process.TerminateProcess

1 0

Re: [Python-Dev] The os module, unix and win32
by Barry Scott Jan. 9, 2004

Jan. 9, 2004

At 08-01-2004 22:11, Guido van Rossum wrote: >(Not sure if this was in response to my "let's do it in Python" post.) Sent before reading you message. An idea who's time has arrived I guess. BArry

1 0

HP-UX clean-up
by Cameron Laird Jan. 9, 2004

Jan. 9, 2004

Python generation for HP-UX is broken. I want to fix it. Do I need to make the case for the first proposition? I think the threading problems are widely recognized ... This isn't an accusation of moral turpitude, incidentally; it just hasn't worked out to solve HP-UX difficulties, for plenty of legiti- mate reasons. I understand that. I happened to try a vanilla generation of 2.3.3 under HP-UX 10.20 yesterday, and encountered multiple non-trivial faults (curses, threading, ...). Experience tells me I'd find roughly the same level of problems with other releases of Python and HP-UX. Do I need to make the case that fixing HP-UX (and eventually Irix and so on) is a Good Thing? I'll assume not, at least for now. I recognize that there's been a lot of churn in HP-UX fixes-- tinkering with threading libraries, compilation options, and so on. I'm sensitive to such. My plan is to make conservative changes, ones unlikely to degrade the situation for other HP-UX users. I have plenty of porting background; I'm sure we can improve on what we have now. Here's what I need: 1. Is it best to carry this on here, or should those of us with HP-UX interests go off by ourselves for a while, and return once we've established consensus? I've been away, and don't feel I know the culture here well. Who *are* the folks with an HP-UX interest? 2. Does anyone happen to have an executable built that can "import Tkinter"? That's my most immediate need. If you can share that with me, it'd help my own situation, and I'll continue to push for corrections in the standard distribution, anyway. I can compile _tkinter.c "by hand", but (more details, later--the current point is just that stuff doesn't work automatically ... 3. I'm very rusty with setup.py. While I was around for its birth, as I recall, I feel clumsy with it now. Is there a write-up anyone would recommend on good setup.py style?

13 36

Re: [Python-Dev] HP-UX clean-up
by Nick Monyatovsky Jan. 9, 2004

Jan. 9, 2004

>------------------- Message from Skip Montanaro -------------------- > > Martin> 2. Assuming -Wl,+s -Wl,+k are warnings: ... > > They cause the +s and +k options to be passed to the linker. They could > also be written as -Wl,+s,+k. > > Martin> 4. On the model selection options, I don't quite understand > Martin> why you say that this set of options gives portable binaries. > Martin> I would have thought that +DA1.1 +DS2.0 is the set of option > Martin> that gives portable binaries. > > I found an HP-UX machine I could log into which has aCC installed. It's > just a 10.20 machine. HP's are on the way out here. I didn't found any > running HP-UX 11. > > About +DA and +DS the aCC man page has this to say: > > +DAarchitecture > Generate code for a particular version of the PA- > RISC architecture specified. Also specifies which > version of the HP-UX math library to link when you > have specified -lm. > > Note Object code generated for PA-RISC 2.0 will > not execute on PA-RISC 1.1 systems. > > To generate code compatible across PA-RISC 1.1 and > 2.0 workstations and servers, use the +DAportable > option. For best performance use +DA with the > model number or architecture where you plan to > execute the program. See the file > /opt/langtools/lib/sched.models for a list of > model numbers and their PA-RISC architecture > designations. > > If you do not specify this option, the default > object code generated is determined automatically > as that of the machine on which you compile. > > Examples > +DA1.1 > +DA867 > +DA2.0 > +DAportable > > The first two examples generate code for the PA- > RISC 1.1 architecture. The third example generates > code for the PA-RISC 2.0 architecture. The fourth > example generates code compatible across PA-RISC > 1.1 and 2.0 workstations and servers. > > +DSmodel Use the instruction scheduler tuned to the model > specified. If this option is not used, the > compiler uses the instruction scheduler for the > architecture on which the program is compiled. > The architecture is determined by uname() (see > uname(2)). model can be a model number, PA-RISC > architecture designation or PA-RISC processor > name. See the file > /opt/langtools/lib/sched.models for a list of > model numbers and processor names. > > Obviously, due to the age of the OS on this machine, there's no mention of > Itanium. > > >> Maybe there should also be some sort of option for expects who will > >> not want Python to be compiled for generic processor; who might > >> purposefully want to compile Python on their particular hardware and > >> have it fully optimized for it to gain maximum performance. > > Martin> No, no, no. If you want to highly tune your build, you are > Martin> expected to edit the makefile. Providing an explicit option for > Martin> that will place a burden on users who are not able to answer > Martin> difficult build questions (and, trust me, this is the majority > Martin> of the users). > > It would also make the configure script that much more fragile. > > Skip > > > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mon%40abaqus.com >-------------------------------------------------------- As far as architectures are concerned, we can no longer find any PA 1.1 machines. All machines that we deal with happen to be PA 2.0 machines. So far we have had no trouble with binaries compiled for PA 2.0. (We re-distribute Python to users in binary form). We had initially also tried to compile for 1.1 and Portable, but found we that those binaries run much slower: there was about 20% performance improvement for switching from PA 1.1 to PA 2.0. But switching from PA-RISC 32-bit to 64-bit brought no performance improvements at all. Binaries compiled for PA 2.0 in 32-bit run successfully on all current processors: 32-bit RISC, 64-bit RISC, as well as Itanium machines. So PA-RISC 2.0 32-bit seems like a reasonable default. Could other users please weigh in here if this is not your typical setup? --- I also accept the point that configure should have a reasonable default when used with no arguments. And what that particular default is, is not particularly relevant. What's important is that it should pick one of the broad/generic setups we are discussing right now, and not tie itself to a very particular hardware on a single machine. What's also good about this, is that at least it will display which Architecture it has chosen (if you look at the -DA -DS options it picked), and not just compile something silently that might be very unportable. Thank you, -- Nick Monyatovsky -- mon(a)abaqus.com

1 0

Re: [Python-Dev] HP-UX clean-up
by Nick Monyatovsky Jan. 9, 2004

Jan. 9, 2004

>------------------- Message from Martin v. Loewis -------------------- > Nick Monyatovsky wrote: > > Would not it be better if you let the users (people who compile Python) decide > > what they want? > > No, not at all. configure should always give a working installation if > invoked without options. If we have to make decisions for people to > obtain such a working installation - fine. We then have to ask whether > people might have made different decisions, and whether we want to > support them in that decision. In many cases, we found that not > giving choices is the right thing. > > So, for your set of options: > 1. configure should find out automatically how to enable large > file support on a 32-bit machine. In fact, there is code in > configure.in to automatically detect LFS. I'm surprised that > you are specifying LFS on the command line - this should > not be necessary. > > 2. Assuming -Wl,+s -Wl,+k are warnings: We would turn them on > if they are always acceptable, and leave them off if there > are cases were they are not applicable. Figuring out whether > warning options are needed/possible is too much effort, > letting the user make the choice is pointless - our other > compilers give plenty of warnings, so I believe that the > HP compiler's complaints are likely false positives or > otherwise irrelevant. > > 3. We would compile/link with threads always if possible, > unless --disable-threads was given. If compiled with threads, > configure should know to pass -lpthread. > > 4. On the model selection options, I don't quite understand > why you say that this set of options gives portable binaries. > I would have thought that +DA1.1 +DS2.0 is the set of option > that gives portable binaries. Of course, if we can assume > all hardware is PA 2.0 meanwhile, this would be fine. In > that case, we would always add the PA 2.0 option set, > leaving the user no choice. > > > 1) It is hard (== unreliable) to try to distinguish them using 'model', > > etc.; > > Then there should be no choice at all. > > > 2) Reliance on such techniques will inevitably break overtime when the new > > processor denominations come into play; > > Then there should be as few options as possible, assuming that the > compiler defaults will do the right thing, anyway (which appears > to be the case with respect to model selection). > > > 3) Most importantly, it is possible to be on the 64-bit machine and be > > purposefully compiling for a 32-bit processor (to cover the broadest customer > > base). It is also possible to be on RISC and compile for ITANIUM and vice > > versa, because it is the same compiler. Moreover, ITANIUM machines will run > > RISC binaries. > > I don't think it should be directly supported. By default, configure > should force PA binaries on PA hardware, and Itanium binaries on Itanium > hardware. > > One issue is whether we should produce 32-bit or 64-bit binaries on > PA 2.0 machines by default; I don't know which one is more reasonable. > Switching to the other one should be as simple as setting CC before > invoking configure (this is how it works on Solaris). > > > People, on the other hand, usually know what machine they are working on > > now, and what machine they want Python to work on after they built it. > > People should be able to specify things that they reasonably should > be able to specify. If they don't specify anything, something reasonable > should still happen. > > > Maybe there should also be some sort of option for expects who will not want > > Python to be compiled for generic processor; who might purposefully want to > > compile Python on their particular hardware and have it fully optimized for it > > to gain maximum performance. > > No, no, no. If you want to highly tune your build, you are expected to > edit the makefile. Providing an explicit option for that will place > a burden on users who are not able to answer difficult build questions > (and, trust me, this is the majority of the users). > > Regards, > Martin >-------------------------------------------------------- Those are all correct observations: we are using an older (2.0) version of Python, and had to add large file support and threads explicitly. I will answer the architectures issue in another message, where people brought more details of this to the surface. Thank you, -- Nick Monyatovsky -- mon(a)abaqus.com

1 0

Re: [Python-Dev] HP-UX clean-up
by Nick Monyatovsky Jan. 9, 2004

Jan. 9, 2004

>------------------- Message from Skip Montanaro -------------------- > > Andrew> My lack of experience with configure make it hard to submit a > Andrew> patch, so I'll ask python-dev first. > > That's okay. What you gave is a good start. I have a couple questions > though: > > 1. How can I distinguish the three processors you described (HP/RISC > 32-bit, HP/RISC 64-bit, HP/ITANIUM 64-bit)? I suspect uname might > be the culprit. If so, can you provide uname output for the three > processors? > > 2. Is there no way to actually create a truly processor-independent > executable (perhaps by creating a "fat" binary)? > > Thx, > > Skip > > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mon%40abaqus.com >-------------------------------------------------------- Would not it be better if you let the users (people who compile Python) decide what they want? There are several reasons for this: 1) It is hard (== unreliable) to try to distinguish them using 'model', etc.; 2) Reliance on such techniques will inevitably break overtime when the new processor denominations come into play; 3) Most importantly, it is possible to be on the 64-bit machine and be purposefully compiling for a 32-bit processor (to cover the broadest customer base). It is also possible to be on RISC and compile for ITANIUM and vice versa, because it is the same compiler. Moreover, ITANIUM machines will run RISC binaries. In other words, the confusion never stops ... People, on the other hand, usually know what machine they are working on now, and what machine they want Python to work on after they built it. There will be less surprises for them of the sort "Oh, that what happened during the compilation!... I had not idea ..." after the fact that you built this, distributed it to somebody, and had report back that it all failed. Maybe there should also be some sort of option for expects who will not want Python to be compiled for generic processor; who might purposefully want to compile Python on their particular hardware and have it fully optimized for it to gain maximum performance. Thank you, -- Nick Monyatovsky -- mon(a)abaqus.com

4 3