Python-Dev July 2001

python-dev@python.org

88 participants
199 discussions

by Martin Sj�gren

Hello I've been writing quite a few mails lately, all concerning import problems. I thought I'd write a little longer mail to explain what I'm doing and what I find strange here. Basically all (at least the 10-20 ones I've checked) the C modules in the distribution have one thing in common: if something in their initFoo() function fails, they return without freeing any memory. I.e. they return an incomplete module. The only way I can think of that one of the standard modules could fail is when you're out of memory, and that's kinda hard to simulate, so I put in a faked failure, i.e. I raised an exception and returned prematurely (in one of my own C modules, not one in the distribution!). The code looks like this: PyErr_SetString(PyExc_ImportError, "foo"); return; /* do other things here, this "fails" */ >>> import Foo Traceback (most recent call last): File "<stdin>", line 1, in ? ImportError: foo >>> import Foo >>> dir() ['Foo', '__builtins__', '__doc__', '__name__'] Huh?! How did this happen? What is Foo doing there? Even more interesting, say that I create a submodule and throw in a bunch of PyCFunctions in it (I stole the code from InitModule since I don't know how to fake submodules in a C module in another way, is there a way?). I create the module, fail on inserting it into the dictionary and DECREF it. Now, that ought to free the darn submodule, doesn't it? Anyway, I wrote a simple "mean" script to test this: try: import Foo except: import Foo while 1: try: reload(Foo) except: pass And this leaks memory like I-don't-know-what! What memory doesn't get freed? Now to my questions: What exactly SHOULD I do when loading my module fails halfway through? Common sense says I should free the memory I've used and the module object ought to be unusable. Why-oh-why can I import Foo, catch the exception, import it again and it shows up in the dictionary? What's the purpose of this? How do I work with submodules in a C module? I find the import semantics really weird here, something is not quite right... Regards, Martin Sjögren -- Martin Sjögren martin(a)strakt.com ICQ : 41245059 Phone: +46 (0)31 405242 Cell: +46 (0)739 169191 GPG key: http://www.strakt.com/~martin/gpg.html

19 years, 7 months

Breakage in CVS on SF?

by barry＠zope.com

Has anybody noticed this problem? % cvs -q up -P -d P Mac/Lib/findertools.py cvs [update aborted]: cannot open .new.findertoo: Permission denied write stdout: Broken pipe

19 years, 7 months

Problems building info documentation

by Charles G Waldman

I did a CVS update and am trying to rebuild the info docs, and am getting the following errors. Any suggestions (other than the obvious one of rewriting html2texi.pl as html2texi.py)? cd /home/cgw/Python/python/dist/src/Doc/info/ make -k ../tools/mkinfo ../html/api/api.html perl -I/home/cgw/Python/python/dist/src/Doc/tools /home/cgw/Python/python/dist/src/Doc/tools/html2texi.pl /home/cgw/Python/python/dist/src/Doc/html/api/api.html /usr/lib/perl5/site_perl/5.6.1/HTML/Element.pm:2091: function main::collect_if_text expected 3 arguments, got 5: Front Matter 1 1 HTML::Element=HASH(0x81bf70c) 0 make: *** [python-api.info] Error 255 ../tools/mkinfo ../html/ext/ext.html perl -I/home/cgw/Python/python/dist/src/Doc/tools /home/cgw/Python/python/dist/src/Doc/tools/html2texi.pl /home/cgw/Python/python/dist/src/Doc/html/ext/ext.html /usr/lib/perl5/site_perl/5.6.1/HTML/Element.pm:2091: function main::collect_if_text expected 3 arguments, got 5: Front Matter 1 1 HTML::Element=HASH(0x81bf6b8) 0 make: *** [python-ext.info] Error 255 ../tools/mkinfo ../html/lib/lib.html perl -I/home/cgw/Python/python/dist/src/Doc/tools /home/cgw/Python/python/dist/src/Doc/tools/html2texi.pl /home/cgw/Python/python/dist/src/Doc/html/lib/lib.html /usr/lib/perl5/site_perl/5.6.1/HTML/Element.pm:2091: function main::collect_if_text expected 3 arguments, got 5: Front Matter 1 1 HTML::Element=HASH(0x81bf76c) 0 ...

19 years, 7 months

PEP for adding a decimal type to Python

by Michael McLay

PEP: XXX Title: Adding a Decimal type to Python Version: $Revision:$ Author: mclay(a)nist.gov <mclay(a)nist.gov> Status: Draft Type: ?? Created: 25-Jul-2001 Python-Version: 2.2 Abstract Several PEPs have been written about fixing Python's numerical types. The proposed changes raise issues about breaking backwards compatibility in the process. Changing the existing numerical types can be avoided by introducing a decimal number type. This change will also enhance the utility of Python for several key markets. A decimal type is also a natural super-type of both integers and floating point numbers. This makes it an important root type for an inheritance tree of numerical types. This PEP suggests adding the decimal number type to Python in such a way that the existing number types will be the default type for .py files and the python command and the new decimal number type will be used for .dp files and the dpython command. Rationale Conflicts surface in the discussion of language design when programming goals differ. One example of this is found when selecting the best method for interpreting numerical values. The correct answer is dependent on the application domain of the software. While Python is very good at providing a simple generalized language, it is not an ideal language in all cases. For developers of scientific application the use of binary numbers, are often important for performance reasons. The developers of financial application need to use decimal numbers in order to control roundoff errors. Decimal numbers are also best for newbie users because decimal numbers have simpler rules and fewer surprises. The current implementation of numbers in Python is limited to a binary floating point type (both imaginary and real) and two types of integers. This makes the language suitable for scientific programming. Python is also suitable for domains which do not make use of numerical types. Changing the existing python implementation to use decimal numbers and the default type for literals is likely to irritate scientific programmers. Having to use special notation for decimal literals will make financial application developers second class citizen. Both groups can coexist and share compiled modules by making the parser of Python sensitive to the context of the syntax. This can be done by adding a new decimal type and then selectively changing the definition of default literals (that is a literal without a type suffix). In the proposed implementation the .py files and the python command would continue to parse numerical literals as they currently are interpreted. The new decimal type would be used for number literals for .dp files and the dpython command. Proposal A new decimal type will be added to Python. The new type will be based on the ANSI standard for decimal numbers. The proposal will also add two new literal for representing numbers A decimal literal will have a 'd' appended to the number and a float literal or an integer literal will have a 'f' appended to the number. The current '.py' file and the use of the python command will continue to use the existing float and integer types for the number literals without a suffix. The proposed change will add support for a second file type with a '.dp' suffix. There will also be an alternative command name, 'dpython', for the Python executable. The decimal number will be used for the interpretation numerical literals in a '.dp' file and when using the 'dpython' command. The following examples illustrate the two commands. $ ./dpython Python 2.2a1 (#87, Jul 26 2001, 11:07:58) [GCC 2.96 20000731 (Linux-Mandrake 8.0 2.96-0.48mdk)] on linux2 Type "copyright", "credits" or "license" for more information. >>> type(21.2) <type 'decimal'> >>> type(21.2f) <type 'float'> >>> type(21f) <type 'int'> >>> 21.2f 21.199999999999999 >>> 21.2 2.12 >>> 1f/2f 0 >>> 1/2 0.5 >>> $./python Python 2.2a1 (#87, Jul 26 2001, 11:07:58) [GCC 2.96 20000731 (Linux-Mandrake 8.0 2.96-0.48mdk)] on linux2 Type "copyright", "credits" or "license" for more information. >>> type(21.2) <type 'float'> >>> type(21.2f) <type 'float'> >>> type(21.2d) <type 'decimal'> >>> 1/2 0 >>> 21.2 21.199999999999999 >>> 21.2d 21.2 The new decimal type is a "super-type" of float, integer, and long, so when decimal math is used there are only decimal numbers, regardless of whether it is an integer or a floating point number. Newbies and developers of financial applications would use the dpython command and the '.dp' suffix for modules. The language will remain unchanged for existing programs. The addition of a decimal type that can be sub-classed may eliminate the need to add inheritance to float or integer types. The inheritance from float and integer are likely to be challenging. How will the inheritance from the float or integer type work? The definition and implementation of these types are dependent on the C compiler used to compile the interpreter. By contrast, a new decimal type could be designed to be highly customizable. The implementation could be implemented like class instances with a dictionary that starts out with three members, a sign, a coefficient, and an exponent. This basic type could be extended with flags that set the type of rounding to be used, or by adding a member that sets the precision of the numbers, or perhaps a minimum and maximum value member could be added. Adding the new file type is also an opportunity to fix some other ugliness in Python. The tab character could be eliminated in from block indentation. The default character type could be set to Unicode. (In dpython a 'b' would be added to the front of strings that are sequences of bytes.) Using Unicode as the default has one important downside. The change would limit the viewing of the '.dp' files to display devices that are Unicode enabled. This may have been a problem five years ago. Would it be today? --- need to add other improvement that could be done in dpython --- Backwards Compatibility The proposed change is backward compatible with the existing syntax when the python command is used. The new dpython command would be used to take advantage of the new language syntax. The python command will have access to the decimal number type and the dpython command will have access to the traditional float and integer types. Both versions of the language could be used to write exactly the same programs that generate exactly the same byte code output. The only difference will be a few syntax improvements in the dpython language. Prototype Implementation An implementation of this PEP has been started, but has not been completed. The parsing works as described, and a partial implementation of a decimal type has been started. The prototype implementation of the decimal object is sufficient for testing the approach of mingling dpython and python. The design of the current implementation does not support sub-classing. This minimal implementation of a decimal type could be completed with a days work. The development of an extendable type, as was described above, could take place in a later release. The interpretation of number literal that does not have a suffix is determined in in the parsetok() function. The function adds a 'd' or 'f' flag to any numerical literal that does not already have a number type suffix. The suffix attached to the numerical literal is based on the command used to invoke the parser or the suffix of the filename. The parsenumber() function in compile.c file was modified to key off the number type suffix. This type indicator is used in a switch statement for compiling the text of the literal into to the correct type of number. The implementation of the decimal type was created by copying the complexobject.[hc] files and then doing a global replace of the word complex with the word decimal. The PyDecimal_FromString method in decimalobject.c interprets the string encoding of a decimal number correctly and populates the data structure that contains the sign, coefficient, and exponent values of a decimal number. A minimal printing of the decimal number has been enabled. It is hard-coded to just print out a scientific notation of the number. The only operator that works properly at this time is negation operator defined in decimal_neg(). The d_sum() and d_prod() function have been started, but they are very broken. No work has been done on implementing the d_quot() function. The example that shows integer division working properly above was done by editing the output. The format of the echoed decimal number was also edited. When a directory in the path contains a '.dp' module and a '.py' module with the same module name the '.dp' module is used. The prototype implementation is available at http://www.gencam.org/python The implementation has only be tested on Mandrake Linux 8.0. Known Problems and Questions The parsetok.c file was duplicated and renamed to parsetok2.c because the pgen program could not resolve the Py_GetProgramName() function. The dpython repr() function should probably return a number with a suffix of 'd' for decimal types if the module is a '.py' module or if the python command is used. Should the repr() function add the 'f' suffix to float and integer values when accessed from a '.dp' module or the dpython command is used? Common Objections Adding a new type results in more rules to remember regarding the use of numbers in Python. Response: In general the rules for using a the decimal number type will be simpler than the rules governing the current set of numerical types. This should make it easier for newbies to learn the dpython language. The benefits to the users who need a decimal type are significant and the added rules will primarily impact these users. The decimal numbers are more precise, which is essential for some application domains. The decimal number rules will tend to simplify the use of python for these applications. The types used in an application will most likely be selected to match the user's requirements. Crossover between the new decimal types and the classic types will be infrequent. For cases where types must be mixed the language will be explicit. There will be no automatic coercion between the types. Exceptions will be raised if an explicit conversion isn't used. Having two languages will confuse users. Response: This is unlikely to be a problem because there will rarely be a python module that requires both types of numbers. If number types must be mixed in a module the proposed syntax provides an easy method to visually distinguish between the different number types. When types are mixed the choice between python and dpython will probably be dictated by the domain of the application developer. The distinction between python and dpython disappears once the language syntax has been compiled. The only problem that might occur is in recognizing which language version is being used when editing a module. An IDE can minimize the chances of confusion by using different background colors or highlighting schemes to distinguish between the versions of the language. Anyone still using vi on a black and white monitor will just have to remember the name of the file being edited. (Which is probably how they think it should be:-) Shouldn't the root numerical type be a rational type? Response: ??? References [1] ANSI standard X3.274-1996. (See http://www2.hursley.ibm.com/decimal/deccode.html) Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End:

19 years, 7 months

number-sig anyone?

by Skip Montanaro

Dev-ers, I have been guilty of generating as much heat as light the past few days on the subject of integer division (though not quite as much heat as Stephen Horne!). For that I apologize. There are several active PEPs related to various aspects of Python's concept of numbers. Yesterday I found these: S 211 pep-0211.txt Adding A New Outer Product Operator Wilson S 228 pep-0228.txt Reworking Python's Numeric Model Zadka S 237 pep-0237.txt Unifying Long Integers and Integers Zadka S 238 pep-0238.txt Non-integer Division Zadka S 239 pep-0239.txt Adding a Rational Type to Python Zadka S 240 pep-0240.txt Adding a Rational Literal to Python Zadka S 242 pep-0242.txt Numeric Kinds Dubois Today I took a look at http://mail.python.org/mailman/listinfo and could find no math-sig or number-sig mailing list. If Python's number system is going to change in one or more backwards-incompatible I think there may only be one chance to get it right. I think a number-sig mailing list would be a worthwhile forum to discuss these issues. If there's already a group specific to this topic I missed it. Point me and I will start reading archives. Skip

19 years, 7 months

Re: Static method and class method comments

by Paul Prescod

>Kevin Smith wrote: > > I am very glad to see the new features of Python 2.2, but I do have a minor > gripe about the implementation of static and class methods. My issue stems > from the fact that when glancing over Python code that uses static or class > methods, you cannot tell that a method is a static or class method by looking > at the point where it is defined. > ... Agree strongly. This will also be a problem for documentation generation tools, type extraction tools and class browsers. I believe it would be easy to add a contextual keyword > class C: > def static foo(x, y): > print "classmethod", x, y -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook

19 years, 7 months

shouldn't we be considering all pending numeric proposals together?

by Skip Montanaro

There are several active or could-be-active PEPs related to Python's numeric behavior: S 211 pep-0211.txt Adding A New Outer Product Operator Wilson S 228 pep-0228.txt Reworking Python's Numeric Model Zadka S 237 pep-0237.txt Unifying Long Integers and Integers Zadka S 238 pep-0238.txt Non-integer Division Zadka S 239 pep-0239.txt Adding a Rational Type to Python Zadka S 240 pep-0240.txt Adding a Rational Literal to Python Zadka S 242 pep-0242.txt Numeric Kinds Dubois Instead of implementing them piecemeal, shouldn't we be considering them as a related group? For example, implementing any or all of PEPs 237, 239 and 240 might well have an effect on what needs to be done for PEP 238. With slight modifications, the proposals in PEP 242 might well subsume PEP 238's functionality in a different way. If the semantics of arithmetic are going to change, I think they should change in the context of expanded capability in the language. -- Skip Montanaro (skip(a)pobox.com) http://www.mojam.com/ http://www.musi-cal.com/

19 years, 7 months

Please have a look at proposed doc changes for time epoch

by Skip Montanaro

I was just reminded by an update to another bug I had submitted that I was assigned bug #434143. Anything that uses time.mktime will fail if the time tuple it is passed is "too old". Unfortunately, by trying to be precise, the message that goes along with the ValueError that's raised, it's a little misleading: ValueError: year out of range (00-99, 1900-*) Obviously, on Unix systems the baseline date is more like 1970. I suspect this error message was written when Python was being developed mostly (or entirely) on Macs. The context in which this arose was a user trying to generate a calendar using the calendar module. https://sourceforge.net/tracker/?func=detail&aid=434143&group_id=5470&atid=… I think this is a difficult problem to solve properly without adding an alternative to time.mktime and making some changes to the various modules that use it (calendar, imaplib and rfc822 in the current CVS tree). I propose instead to make a few documentation changes: * in Modules/timemodule.c, make the error message more vague ;-) * in Doc/lib/lib{time,calendar}.tex indicate that the "epoch" is platform-dependent I'm more than happy to add the necessary ifdefs to Modules/timemodule.c if we can settle on what the actual epochs are for the various platforms. For Unix it is 1970-01-01. For Macs I think it is 1900-01-01. Is it 1904-01-01 on Windows? If you have a moment, please have a look at the above sourceforge url. I'd like to get this off my plate in the next few days. Skip

19 years, 7 months

Branches here, branches there, branches everywherte

by Greg Ward

I've finally started reviewing the changes made to the Distutils during my extended leave-of-absence, and making a few minor commits. So far I've just made these commits on the trunk, because I don't really understand anything else. (Yes, yes, I've read the CVS docs many many times. It just takes a while to sink in.) Am I right in doing this? Ie. will 2.2a2 be released from the trunk? Or should I be doing commits that I want in 2.2a2 on the 22a1 branch? Greg -- Greg Ward - nerd gward(a)python.net http://starship.python.net/~gward/ All of science is either physics or stamp collecting.

19 years, 7 months

Re: [Python-Dev] A future division proposal

by Moore, Paul

> I don't know what the best name or symbol would be > (olddiv, ///?) and admittedly it is ugly to have 3 > division operators. But the alternatives seem far, > far uglier. Isn't this a case where practicality > beats purity (for keywords or operators)? You could probably write a function to do this. There's no need for anything built into Python. Actually, when I tried, I got into a bit of a mess getting the type checks (which you need) right - def olddiv(n,m): if type(n) == type(m) == type(0): return n//m else: return n/m But this needs the checks expanded to take longs into account. Which is where it gets messy. But: a) It can be done, and b) The fact that it's messy probably exposes what's wrong with the old semantics quite well :-) Paul.

19 years, 7 months

← Newer
1
2
3
4
5
6
7
8
9
...
20
Older →

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev July 2001