Python-Dev June 2005

python-dev@python.org

111 participants
115 discussions

by Raymond Hettinger

I'm getting a compiler warning from your checkin: C:\py25\Objects\floatobject.c(1430) : warning C4244: 'initializing' : conversion from 'double ' to 'float ', possible loss of data

15 years, 9 months

Adding Python-Native Threads

by Adam Olsen

There are some very serious problems with threads today. Events are often proposed as a solution to these problems, but they have their own issues. A detailed analysis of the two options and some solutions (most of which are used in my proposal) can be found in Why Events Are A Bad Idea [0]. I'm going to skip the event aspects and sum up the thread problems here: * Expensive (resident memory, address space, time to create and to switch between) * Finely-grained atomicity (python is coarser than C, but still finer than we need) * Unpredictable (switching between them is not deterministic) * Uninterruptible (no way to kill them, instead you must set a flag and make sure they check it regularly) * Fail silently (if they die with an exception it gets printed to the console but the rest of the program is left uninformed) To resolve these problems I propose adding lightweight cooperative threads to Python. They can be built around a Frame object, suspending and resuming like generators do. That much is quite easy. Avoiding the C stack is a bit harder. However, it can be done if we introduce a new operator for threaded calls (which I refer to as non-atomic calls), which I propose be "func@()". As a bonus this prevents any operation which doesn't use the non-atomic call syntax from triggering a thread switch accidentally. After that you need a way of creating new threads. I propose we use a "sibling func@()" statement (note that the "@()" would be parsed as part of the statement itself, and not as an expression). It would create a new thread and run func in it, but more importantly it would also "prejoin" the new thread to the current function. This means that for the current function to return to its parent, all of the threads it created must first return themselves. It also means that an exception in any of them will be propagated to the parent of the creator, after interrupting all of its siblings (with an Interrupted exception) so that it is permitted to return. Now for an example of how this would work in practice, using an echo server. Note that I am assuming a "con" module (short for concurrent) that would contain standard functions to support these python-native threads. from con import bootstrap, tcplisten, ipv6addr, ipv4addr def main(): main_s = tcplisten@(ipv6addr('::1', 6839), ipv4addr('127.0.0.1', 6839)) while 1: sibling echo@(main_s.accept@()) def echo(s): try: try: while 1: s.write@(s.read@()) except (EOFError, IOError): pass finally: s.close() if __name__ == '__main__': try: bootstrap(main) except KeyboardInterrupt: pass And here is a diagram of the resulting cactus stack, assuming three clients, two of which are reading and the other is writing. bootstrap - main - accept | echo - read | echo - read | echo - write Some notes: * An idle@() function would be used for all thread switches. I/O functions such as read@() and write@() would use it internally, and idle would internally call a function to do poll() on all file descriptors. * idle is the function that will raise Interrupted * It should be illegal to attempt a non-atomic call once your thread is in an interrupted state. This ensures you cannot get "interrupted twice", by having something further up the call stack fail and interrupt you while you are already handling an interruption. * Being a cooperative system, it does not allow you to use multiple CPUs simultaneously. I don't regard this as a significant problem, and in any case there are better ways use multiple CPUs if you need to. References: [0] http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_h… -- Adam Olsen, aka Rhamphoryncus

15 years, 9 months

Re: [Python-Dev] Decimal floats as default (was: discussion aboutPEP239 and 240)

by Michael Chermside

Fredrik Johansson writes: > In either case, compatibility can be ensured by allowing both n-digit > decimal and hardware binary precision for floats, settable via a float > context. Perhaps you can show me a design (or working code) that proves me wrong, but I don't believe that such a design could be made compatible with the existing Decimal module... certainly while continuing to maintain compatibility with the Cowlinshaw specification. > There is the alternative of providing decimal literals by using > separate decimal and binary float base types If, by this, you mean adding a "binary float context" modeled after the Decimal float context and providing access to the underlying FP flags and traps and generally enhancing the use of binary FP, then I think it's a great idea. It's probably impossible to write in a cross-platform manner (because C supplies support for binary FP but does not offer access to the flags and traps), but this is one of those few cases where it's worth using platform-and-compiler specific code. Of course, someone still has to step forward and offer to code it. -- Michael Chermside

15 years, 9 months

Weekly Python Patch/Bug Summary

by Kurt B. Kaiser

Patch / Bug Summary ___________________ Patches : 344 open ( +6) / 2875 closed ( +9) / 3219 total (+15) Bugs : 897 open (-17) / 5094 closed (+34) / 5991 total (+17) RFE : 191 open ( +3) / 170 closed ( +0) / 361 total ( +3) New / Reopened Patches ______________________ fileinput openfile patch, bz2fileinput (2005-06-22) http://python.org/sf/1225466 opened by Stephan Boettcher Proposal to implement comment rows in csv module (2005-06-22) http://python.org/sf/1225769 opened by Iain Haslam Patch for (Doc) bug #1217513 (2005-06-25) CLOSED http://python.org/sf/1227442 opened by Peter van Kampen Patch for (Doc) bug #1225705 (2005-06-25) CLOSED http://python.org/sf/1227444 opened by Peter van Kampen Patch for (Doc) bug #1175022 (2005-06-25) CLOSED http://python.org/sf/1227489 opened by Peter van Kampen Patch for (Doc) bug #1186072 (2005-06-25) CLOSED http://python.org/sf/1227495 opened by Peter van Kampen Patch for (Doc) bug #1166582 (2005-06-25) CLOSED http://python.org/sf/1227513 opened by Peter van Kampen Patch for (Doc) bug #1212195 (2005-06-26) http://python.org/sf/1227545 opened by Peter van Kampen Patch for (Doc) bug #1219273 (2005-06-26) http://python.org/sf/1227568 opened by Peter van Kampen Patch for (Doc) bug #1228904 (2005-06-29) http://python.org/sf/1229935 opened by Peter van Kampen solaris 10 should not define _XOPEN_SOURCE_EXTENDED (2005-06-27) http://python.org/sf/1227966 opened by Bill Clarke code.py use sys.excepthook to display exceptions (2005-06-27) http://python.org/sf/1228112 opened by Miki Tebeka optionally allow mutable builtin types (2005-06-28) http://python.org/sf/1229239 opened by Greg Couch optionally allow mutable builtin types (2005-06-28) CLOSED http://python.org/sf/1229280 opened by Greg Couch Zipfile fix create_system information (2005-06-29) http://python.org/sf/1229511 opened by Hendrik Muhs Patches Closed ______________ Building on Mac OS X 10.1 (2004-01-02) http://python.org/sf/869555 closed by mwh httplib mentions getreply instead of getresponse (2005-05-16) http://python.org/sf/1203094 closed by birkenfeld note that os.chown can have -1 as an argument (2005-06-01) http://python.org/sf/1213031 closed by birkenfeld Patch for (Doc) bug #1217513 (2005-06-25) http://python.org/sf/1227442 closed by birkenfeld Patch for (Doc) bug #1225705 (2005-06-25) http://python.org/sf/1227444 closed by birkenfeld Patch for (Doc) bug #1175022 (2005-06-25) http://python.org/sf/1227489 closed by birkenfeld Patch for (Doc) bug #1186072 (2005-06-25) http://python.org/sf/1227495 closed by birkenfeld Patch for (Doc) bug #1166582 (2005-06-25) http://python.org/sf/1227513 closed by birkenfeld cgitb: make more usable for 'binary-only' sw (new patch) (2005-02-19) http://python.org/sf/1144549 closed by birkenfeld optionally allow mutable builtin types (2005-06-28) http://python.org/sf/1229280 closed by gregcouch New / Reopened Bugs ___________________ crash in gcmodule.c on python reinitialization (2005-06-22) http://python.org/sf/1225584 opened by Alexander Miseler os.environ documentation should mention unsetenv (2005-06-22) CLOSED http://python.org/sf/1225705 opened by Brian Wellington SimpleXMLRPCServer example is broken (2004-10-06) http://python.org/sf/1041501 reopened by kjohnson Can't take sequence slice past 32K (2005-06-23) CLOSED http://python.org/sf/1226404 reopened by tim_one Can't take sequence slice past 32K (2005-06-23) CLOSED http://python.org/sf/1226404 opened by Gene Cash cPickle and pickle dump bug (inf float) (2005-06-24) CLOSED http://python.org/sf/1226955 opened by Andrea Bolzonella segfault in os module (2005-06-24) http://python.org/sf/1226969 opened by jacobo_es Queue class does not inherit object (2005-06-24) CLOSED http://python.org/sf/1227166 opened by Vincent Côté-Roy CGIXMLRPCRequestHandler example uses nonexistant "div()" (2005-06-24) CLOSED http://python.org/sf/1227181 opened by Danny Yoo Two requests to one file are not done in parallel (2005-06-25) http://python.org/sf/1227480 opened by Markus Franz subprocess: inheritance of std descriptors inconsistent (2005-06-26) http://python.org/sf/1227748 opened by André Malo shelve/bsddb crash on db close (2005-06-26) http://python.org/sf/1227955 opened by Scott cPickle and pickle dump bug (inf float) (2005-06-27) CLOSED http://python.org/sf/1228015 opened by Andrea Bolzonella usage of type "char*" instead of "const char*" (2005-06-27) http://python.org/sf/1228053 opened by Mattias Ekholm update to 7.15 zlib & 7.18.1 ZipFile Objects docs (2005-06-28) CLOSED http://python.org/sf/1228830 opened by Brian vdB weakref example broken (2005-06-28) http://python.org/sf/1228904 opened by paolo veronelli error whit dict working whith class (2005-06-28) CLOSED http://python.org/sf/1229264 opened by ST2015 No struct.pack exception for some out of range integers (2005-06-29) http://python.org/sf/1229380 opened by Adal Chiriliuc missing Py_DECREF in PyObject_CallMethod (2005-06-28) http://python.org/sf/1229429 opened by Gary httplib error checking. (2005-06-29) http://python.org/sf/1229646 opened by Chris Johns Tracing C function calls and keyword arguments (2005-06-29) http://python.org/sf/1229680 opened by Frank Cieslok Bus error in extension with gcc 3.3 (2005-06-29) http://python.org/sf/1229788 opened by Gary Robinson Bugs Closed ___________ Line endings problem with Python 2.4.1 cause imports to fail (2005-06-21) http://python.org/sf/1225059 closed by doerwalter os.environ documentation should mention unsetenv (2005-06-22) http://python.org/sf/1225705 closed by birkenfeld int/long unification and hex() (2005-06-20) http://python.org/sf/1224347 closed by rhettinger rsplit introduces spurious CR (2005-06-21) http://python.org/sf/1224672 closed by birkenfeld subprocess.Popen fails with closed stdout (2005-04-10) http://python.org/sf/1180160 closed by astrand Can't take sequence slice past 32K (2005-06-23) http://python.org/sf/1226404 closed by mwh Can't take sequence slice past 32K (2005-06-23) http://python.org/sf/1226404 closed by birkenfeld cPickle and pickle dump bug (inf float) (2005-06-24) http://python.org/sf/1226955 closed by mwh Queue class does not inherit object (2005-06-24) http://python.org/sf/1227166 closed by rhettinger CGIXMLRPCRequestHandler example uses nonexistant "div()" (2005-06-24) http://python.org/sf/1227181 closed by dyoo Info from man python not in docs (2005-06-08) http://python.org/sf/1217152 closed by birkenfeld Omission in docs for smtplib.SMTP.sendmail() (2005-06-09) http://python.org/sf/1217513 closed by birkenfeld httplib docs mentioning HTTPConnection.getreply (2005-05-15) http://python.org/sf/1202475 closed by birkenfeld os.waitpid docs don't specify return value for WNOHANG (2005-04-26) http://python.org/sf/1190563 closed by birkenfeld cgi.CGIHTTPRequestHandler remembers QUERY_STRING (2003-09-07) http://python.org/sf/801992 closed by birkenfeld property example code error (2005-04-01) http://python.org/sf/1175022 closed by birkenfeld tempfile.mktemp() omits pid from name (2003-12-10) http://python.org/sf/857566 closed by birkenfeld tempnam doc doesn't include link to tmpfile (2005-04-19) http://python.org/sf/1186072 closed by birkenfeld IterableUserDict not in docs (2005-03-19) http://python.org/sf/1166582 closed by birkenfeld Python keeps file references after calling close methode (2005-04-10) http://python.org/sf/1180237 closed by birkenfeld thisid not intialized in pindent.py script (2004-11-25) http://python.org/sf/1072853 closed by birkenfeld raw_input() displays wrong unicode prompt (2005-01-10) http://python.org/sf/1099364 closed by birkenfeld Cookie.py produces invalid code (2005-01-25) http://python.org/sf/1108948 closed by birkenfeld Cross-site scripting on BaseHTTPServer (2005-01-11) http://python.org/sf/1100201 closed by birkenfeld urllib2.py assumes 206 is an error (2005-04-07) http://python.org/sf/1178145 closed by birkenfeld poorly named variable in urllib2.py (2005-04-03) http://python.org/sf/1175848 closed by birkenfeld self.length shield exception in httplib (2005-03-03) http://python.org/sf/1155638 closed by birkenfeld BaseCookie does not call value_decode (2005-03-25) http://python.org/sf/1170279 closed by birkenfeld doctest.script_from_examples() result sometimes un-exec-able (2005-03-29) http://python.org/sf/1172785 closed by birkenfeld Can't read some http URLs using neither urllib, nor urllib2 (2004-11-01) http://python.org/sf/1058059 closed by birkenfeld CGIHTTPServer: directories/scripts with spaces in their name (2004-12-19) http://python.org/sf/1088039 closed by birkenfeld CGI QUERY_STRING (2004-05-02) http://python.org/sf/946247 closed by birkenfeld attempt to access sys.argv when it doesn't exist (2003-11-10) http://python.org/sf/839151 closed by birkenfeld cPickle and pickle dump bug (inf float) (2005-06-27) http://python.org/sf/1228015 closed by mwh update to 7.15 zlib & 7.18.1 ZipFile Objects docs (2005-06-28) http://python.org/sf/1228830 closed by rhettinger error whit dict working whith class (2005-06-28) http://python.org/sf/1229264 closed by birkenfeld New / Reopened RFE __________________ Add Jason Orendorff's path module to stdlib (2005-06-23) http://python.org/sf/1226256 opened by Michael Hoffman

15 years, 9 months

Re: [Python-Dev] Adding the 'path' module (was Re: Some RFEforreview)

by Tony Meyer

Maybe this has already been answered somewhere (although I don't recall seeing it, and it's not in the sourceforge tracker) but has anyone asked Jason Orendorff what his opinion about this (including the module in the stdlib) is? If someone has, or if he posted it somewhere other than here, could someone provide a link to it? =Tony.Meyer

15 years, 9 months

New python developer

by Fabien

Hello, I'm using Python for about 2 years, and I would like to go further in python developement. So that's why I've subscribed to python-dev. And since I'm not very good in C, I think I will try to help and to submit patchs in pure python. -- Fabien SCHWOB (and sorry for my english but it's not my mother tongue)

15 years, 9 months

getch() in msvcrt does not accept extended characters.

by Darryl Dixon

Hi all, Please CC: me on any replies as I am not subscribed to this list. I am the lead maintainer for the ntlmaps project (http://ntlmaps.sourceforge.net). Recently a bugreport logged against ntlmaps has brought an issue to my attention (see: https://sourceforge.net/tracker/index.php?func=detail&aid=1224877&group_id=…). The getpass() function in module getpass does not accept extended characters on Windows. On Windows, the getpass() source uses the msvcrt module to capture one character at a time from stdin via the _getch() function defined in conio.h. Microsoft support capturing extended characters via _getch (see: http://support.microsoft.com/default.aspx?scid=kb;en-us;57888), but it requires making a second call to getch() if one of the 'magic' returns is encountered in the first call (0x00 or 0xE0). The relevant chunk of code in Python that would probably need to be changed to support this appears to be in msvcrtmodule.c: static PyObject * msvcrt_getch(PyObject *self, PyObject *args) { int ch; char s[1]; if (!PyArg_ParseTuple(args, ":getch")) return NULL; Py_BEGIN_ALLOW_THREADS ch = _getch(); Py_END_ALLOW_THREADS s[0] = ch; return PyString_FromStringAndSize(s, 1); } It would seem that it could be made to support extended characters by changing it to: static PyObject * msvcrt_getch(PyObject *self, PyObject *args) { int ch; char s[1]; if (!PyArg_ParseTuple(args, ":getch")) return NULL; Py_BEGIN_ALLOW_THREADS ch = _getch(); + if (ch == 0x00 || ch == 0XE0) + ch = _getch(); Py_END_ALLOW_THREADS s[0] = ch; return PyString_FromStringAndSize(s, 1); } I haven't yet read the code for PyString_FromStringAndSize(), but presumably it can coerce unicode/extended results out of the int (or could be made to)? What are the chances of modifying msvcrt_getch() to support extended chars? The unix_getpass() seems to already work OK with extended characters, so this would ~seem~ to be a logical extension to the Windows version. Thoughts? -- Darryl Dixon <esrever_otua(a)pythonhacker.is-a-geek.net>

15 years, 9 months

PySWT -- Python binding of SWT using GCJ + SIP

by Zilin Du

Hi, all, I just made an initial Python binding for SWT library from IBM's Eclipse project using GCJ + SIP. you can find some information here: http://www.cs.nyu.edu/zilin/pyswt/pmwiki.php The basic idea is as follows: 1. use GCJ compile SWT source codes 2. use gcjh to generate C++ header files for SWT 3. I modified SIP's code generator to generate C++ wrapper codes. It is still in very early stage, but most widgets are usable now. Hope I can get more time work on this. Weclome any feedbacks, and feel free to forward this annocement to any place you want@ thanks! Zilin

15 years, 9 months

Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE forreview)

by Anders J. Munch

> From: Guido van Rossum [mailto:gvanrossum@gmail.com] > > >>> datetime.datetime.utcfromtimestamp(time.time()) - datetime.datetime.utcnow() > datetime.timedelta(0) I overlooked the utcfromtimestamp method, sorry. > Your bug is similar to comparing centimeters to inches, or speed to > acceleration, or any number of similar mistakes. Quite so, and that is exactly the point. time.time() unambiguously identifies a point in time. A datetime object does not. At least not unless a tzinfo object is included, and even then there is a corner case at the end of DST that cannot possibly be handled. If ctime/mtime/atime were to return datetime objects, that would pretty much have to be UTC to not lose information in the DST transition. I doubt that's what Walter wanted though, as that would leave users with the job of converting from UTC datetime to local datetime; - unless perhaps I've overlooked a convenient UTC->local conversion method? - Anders

15 years, 9 months

Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE forreview)

by Anders J. Munch

> I wrote: > > > Alas datetime objects do not unambiguously identify a point in time. > > datetime objects are not timestamps: They represent the related but > > different concept of _local time_, which can be good for > presentation, > > but shouldn't be allowed anywhere near a persistent store. > GvR replied: > You misunderstand the datetime module! You can have a datetime object > whose timezone is UTC; or you can have a convention in your API that > datetime objects without timezone represent UTC. I can do a lot of things in my own code, and I'm sure the datetime module provides tools that I can build on to do so, but that doesn't help in interfacing with other people's code -- such as the standard library. Try saving a pickle of datetime.now() and reading it on a computer set to a different time zone. The repr will then show the local time on the _originating_ computer, and figuring out the absolute time this corresponds to requires knowledge of time zone and DST settings on the originating computer. How about datetime.utcnow(), then? Just use UTC for datetime timestamps? But that goes against the grain of the datetime module: Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import datetime, time >>> datetime.datetime.fromtimestamp(time.time()) - datetime.datetime.now() datetime.timedelta(0) >>> datetime.datetime.fromtimestamp(time.time()) - datetime.datetime.utcnow() datetime.timedelta(0, 7200) It seems when I subtract the present time from the present time, there's a two hour difference. - Anders

15 years, 9 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev June 2005