[Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not?

Alexander Belopolsky alexander.belopolsky at gmail.com
Wed Jul 7 17:29:19 CEST 2010


On Tue, Jul 6, 2010 at 11:54 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 7/6/2010 3:59 PM, Alexander Belopolsky wrote:
>
> I am more interested in Brett's overall vision than this particular module.
> I understand that to be one of a stdlib that is separate from CPython and is
> indeed the standard Python library.
>

I am also very much interested in the overall vision, but I would like
to keep the datetime.py thread focused, so I a going to reply to broad
questions under a separate subject.

> Questions:
>
> 1. Would the other distributions use a standard stdlib rather than current
> individual versions?

I certainly hope they will.  In the ideal word, passing test.regrtest
with unmodified Lib should be *the* definition of what is called
Python.  I understand that there is already some work underway in this
direction such as marking implementation specific tests with
appropriate decorators.

>
> 2. Would the other distributions pool their currently separate stdlib
> efforts to help maintain one standard stdlib?

I believe that making stdlib and test.regrtest more friendly to
alternative implementations will go long way towards this goal.  It
will, of course, be a decision that each project will have to make.

>
> 3. What version of Python would be allowed for use in the stdlib? I would
> like the stdlib for 3.x to be able to use 3.x code. This would be only a
> minor concern for CPython as long as 2.7 is maintained, but a major concern
> for the other implementation currently 'stuck' in 2.x only. A good 3to2
> would be needed.

Availability of python equivalents will hopefully help  "other
implementation currently 'stuck' in 2.x only" to get "unstuck" and
move to 3.x.   I understand that this is a somewhat sensitive issue at
the moment, but I believe a decision has been made supporting new
features for 2.x is outside of python-dev focus.


> 4. Does not ctypes make it possible to replace a method of a Python-coded
> class with a faster C version, with something like
>  try:
>    connect to methods.dll
>    check that function xyx exists
>    replace Someclass.xyy with ctypes wrapper
>  except: pass
> For instance, the SequenceMatcher heuristic was added to speedup the
> matching process that I believe is encapsulated in one O(n**2) or so
> bottleneck method. I believe most everything else is O(n) bookkeeping.
>
The ctypes modules is very CPython centric as far as I know.   For the
new modules, this may be a valid way to rapidly develop accelerated
versions.   For modules that are already written in C, I don't see
much benefit in replacing them with ctypes wrappers.


> [.. datetime specific discussion skipped ..]
> From scanning that and the posts here, it seems like a pep or other doc on
> dual version modules would be a good idea. It should at least document how
> to code the switch from python version to the x coded version and how to
> test both, as discussed.
>
I am certainly not ready to write such PEP.   I may be in a better
position to contribute to it after I gain more experience with
test_datetime.py.   At the moment I have more questions than answers.

For example, the established practice appears to be:

modulename.py

# Python code

try:
    from _modulename import *
except:
    pass

This is supposed to generate a .pyc file with no python definitions in
it if  _modulename is available.  The problem with datetime.py is that
it have several helper methods like _ymd2ord() that will still stay in
the module.  Should an "else:" clause be added to clean these up?
should these methods become class or static methods as appropriate?

The  established practice for testing is

py_module = support.import_fresh_module('modulename', blocked=['_modulename'])
c_module = support.import_fresh_module('modulename', fresh=['_modulename'])

class TestDefnitions: # not a unittest.TestCase subclass
       def test_foo(self):
            self.module.foo(..)
       ...

class C_Test(TestDefnitions, unittest.TestCase):
       module = c_module

class Py_Test(TestDefnitions, unittest.TestCase):
       module = py_module


For datetime.py this approach presents several problems:

1. replacing datetime with self.module.datetime everywhere can get
messy quickly.
2. There are test classes defined at the test_datetime module level
that subclass from datetime classes.  The self.module is not available
at the module level.  These should probably be moved to setUp()
methods and attached to test case self.
3. If #2 is resolved by moving definitions inside functions, the
classes will become unpickleable and pickle tests will break.  Some
hackery involving injecting these classes into __main__ or module
globals may be required.

These challenges make datetime.py an interesting showcase for other
modules, so rather than writing a PEP based on abstract ideas, I think
it is better to get datetime.py integrated first and try to establish
the best practices on the way.


More information about the Python-Dev mailing list