[Python-Dev] PyString_GET_SIZE()

[Guido van Rossum]

> Hmm. Is there a good place to start listing these todo items? One that we
> can truly use for communicating this info? In Apache, we have a file named
> STATUS that everybody uses for dropping ideas, patch references, critical
> bugs, etc. As that file gets changed, we see it in the -checkins alias, so
> everybody is aware of the suggested changes/problems/available patches/etc.
> It also holds people's votes on particular changes.
> 
> Can we institute something similar? Possibly Misc/STATUS? Should I post
> Apache's STATUS file as an example?

Possibly, but I'm somewhat skeptical.  I used to have a large TODO
file -- still have it -- but it's so full of long-term ideas that
never happened that I rarely look in it any more.  I never got into
the discipline of using it for my day-to-day priorities -- it was more
of a place to write down long-term ideas so I could forget about
them.  I'll append it.  I appreciate a copy of Apache's STATUS file.

--Guido van Rossum (home page: http://www.python.org/~guido/)

NEW TODO LIST FOR 1.6
=====================

IDLE: set window class?
socket/ssl version mystery

UNIFIED TODO LIST
=================

From psa-members post
---------------------

Documentation.  I am working on api.tex.  The ext.tex file needs a
major overhaul too -- it's rather out of date.  I still haven't gotten
Harlequin's webmaker to produce decent HTML from the FrameMaker
version of the reference manual.

Unbundling.  I'm thinking of unbundling the documentation, demos, most
tools, and some platform specific or rarely used extension modules.  A
proposal would be welcome.

There are some unresolved issued with the Py_NewInterpreter()
interface and cleaning up threads, e.g. Demo/threads/condition.py
doesn't work.  I think I may have to change some interfaces (arguably
this is more than just a bug).

Some stuff that I have pending (the really small or well understood
bugs left out):

- urllib or ftplib seems to be getting the second transfer not quite
right; needs investigation

- tmb submitted patches to make creating large lists by appending
items one at a time faster; these were controversial because they work
by doubling the allocation size, which may waste precious mmmory for
very large lists; it is not clear whether it's actually needed

- my attempts at threading _tkinter backfired; the Tcl group says I
have to build my own version of Tcl and write my own notifier; but I
don't want to!

- and what to do about createfilehandler on Win and Mac platforms with
Tk 8.0?

- repr(), str(), print and cmp() may run into infinite loops when
comparing recursive objects; this should be fixed by checking the
recursion limit present in the thread state

- when the byte code gets too big, a better warning should be issued;
I've seen it dump core

- I'm not sure that dict objects always propagate exceptions in
comparisons properly

- there's discussion about __init__.py when using ni; should it be a
submodule of the project or run in the project's module?

- more places should accept arbitrary sequences or mappings instead of
lists/tuples and dicts

- there's a bunch of typedefs (e.g. binaryfunc) that should be grandly
renamed

- the .h files should all be renamed

- some .c files in Modules are better moved to Python now

- doc strings everywhere?

- some modules still break on 64-bit platforms

- sometimes test_select fails on Win (NT or 95?) with 'error 0'


1.5 TODO list
=============

urllib ftp problems -- see message from Wolfgang Weitz

finish api.tex! especially, document new threading/embedding interface

fix ref.doc

look into tmb's diffs (perhaps with an ifdef?)

make cPickle use new PyImporty_Import()

Showstoppers
------------

Plain old bugs
--------------

problem in _tkinter when passing a non-str'able object to tk.call()

check photoimage bug reported by Roman Milner to psa-members

GetAttr/SetAttr/DelAttr don't check for string object

"Must do" wishes
----------------

check for NULL from Py_Initmodule*()

limit recursion in repr() etc. a la stack check

error when bytecode gets too big is confusing (even core dumps possible!)

catch and propagate errors in comparisons in dictobject.c

Thread problems
---------------

Windows 95/NT
-------------

Tk 8.0 compatibility -- what do do about CreateFileHandler?

seeks in mailbox.py cause it not to work with files opened in text mode

??? select test fails on Windows with 'error 0' (is this w95?)

64-bit problems
---------------

verify that (2L**3671-1)%107423446849988119 is gone

test_audioop gives loads of "unaligned access" errors but reports OK

test_rgbimg and test_imageop gives segmentation faults

Other platform specific
-----------------------

mac Tk 8.0 compatibility -- what do do about CreateFileHandler?

AIX and linux have broken strftime

ultrix needs -li (i18n library) to use threads

FAQ
---

officially discard the python FAQ from usenet list of FAQs

add copyright boilerplate to FAQ (???)

point to faq wiz on website

point faq maintainers to faq wiz

Documentation
-------------

\code{~connolly} in liburllib.tex loses the ~ in html conversion

add __doc__ strings to type objects (and everywhere else :-)

PC/example_nt should also support DevStudio VC++ 5.0

restructure tutorial according to Skip M's suggestions

@e in HTML generated from libregex.tex.

Cleanup
-------

get rid of rgbimg and imgop (and audioop?)

move Demo and (some?) Tools from distribution to ftp/web site?

merge objimpl.h into object.h; clean up the comments

a bunch of typedefs (e.g. binaryfunc) in object.h are still old style;
also things like BAD_STATIC_FORWARD, staticforward, statichere (and
all the configure macros of course -- HAVE_*)

use size_t more often (e.g. for all string and list lengths)
(problem: can't be negative)

more routines should take arbitrary sequences -- e.g. string.join(), apply()

unified uu unpacking of image files for tests

urllib ftp cache behaves strangely when files are not read till EOF

Wishes
------

improve finalization order in modules

need portable file locking primitives

look into Embed.Lutz example

add fileevent to Tkinter

better output when regrtest detects an output difference

better diagnostic for skipping tests of optional features

Get rid of all references to old cgi module from web site

better unified drivers for uu, quopri, base64, binhex, and mimify

ni.py interferes with Mark Hammond's special module location mods.

ni.py and __init__.py?

argcount errors in Tk callbacks show up in traceback for mainloop(n).

Done/fixed/dealt with
---------------------

[fixed] copy.py bug (use of temp objects in memo?)

[that was on purpose] (dl_nt.c lost some code :-)

[done] see if method objects can be allocated from a free list

[done] the PC/example_nt readme file should be updated

[done] document new behavior of os.environ

[done] embedding example

[done] remove old FAQ from distribution; move new FAQ.html in

[done, I presume] AIX 4.2 supports ld -G

[fixed] test_strftime crashes on Mac with float -> int conversion

[never mind] report of 'str' not defined in NT freeze sometimes

[done] build debug and release versions like customary

[done] get 5.0 compiler to work; or reinstall 4.2 compiler

[done] rename mapping*c to dict*c in project file

[fixed] cgi.py uses nonportable unlink semantics

[fixed] bug I found in sortmail.py regexes

[done] look into JimF's PyImport_Import (to work with ni and rexec)

[done] generate ref/postscript

[fixed] test_strftime for windows

[done] get new cPickle

[done] Jim F's cgi.py and tempfile.py (?)

[done] Jack's diffs

[fixed] __del__ fails if __builtins__ is already gone

[done] Lee Busby's patches

[no; mhammond has dropped his dialog.py] Lib/tkinter/Dialog.c should
be renamed to avoid conflict with PythonWin's Dialog

[fixed] fileno() is undocumented

[done] Should add HAVE_KILL to configure.in

[never mind] look into greg's comments about thread state

[fixed] After running test_thread interactively, my sys.exitfunc()
dies: Fatal Python error: eval_code2 called without a current thread
IOT trap

[done] add sys.executable == path of Python binary

[done] add SocketType to socket module

[done] in regrtest, wrap __import__(test) in try/except+traceback

[no; rippped out] is the last_name trick in dictobject.c still needed?

[taken care of] delete _ on EOF for better GC?

[done] _tkinter threading

[fixed] Leak in: while 1: a=eval("1") (also in while 1: exec "a=1")

[fixed] urlparse clearcache makes the module non reentrant!

[explained] leak in exec reported by Just

[yes] make sure Lemburg's regex complaint is taken care of

[done] cgi.py should be runnable without environment or input

[fixed] shouldn't ignore exceptions in comparisons

[probably fixed] exception in ftplib.close can trip urllib

[fixed] PyFile_WriteString masks exceptions!

[fixed] test_audioop fails

[fixed] test_pwd uses an unreliable way to pick a non-existing uid for
fakeuid (if the largerst uid is the largest unsigned short, adding 1
makes it 0)

[fixed] object.[ch] doesn't have PyObject_HasAttr() even though
abstract.h has it

[fixed] win32 should guess sys.path like Unix getpath.c

[fixed] [by using sleep 5] DG/UX fails test_signal

[fixed] getpath.c doesn't work on DOS or OS/2 (Jim A submitted getpathp.c)

[fixed] use binary mode in ftpmirror script

[fixed] unlink before rename in ftpmirror script

[fixed] complexobject.c still has the c_error global

[fixed] ihooks' __import__ is leaking file descriptors (cause: the following)

[fixed] catching exceptions leaks frames!!! (same as following)

[fixed] ceval.c :: set_exc_info() appears to have a refcount problem
w.r.t. tstate->exc_* (it never does a DECREF on the prior contents).

[fixed] socket test fails on Windows in os.fork()

[fixed] add PIL init to tkappinit.c

[fixed] cPickle is broken

[fixed] Try curses and ncurses for readline (make it a loop?)

[fixed] make sure dis.py is up to date.

[fixed] read() of 8M file should be more efficient on Windows...

[fixed] regexpr.c: '^' doesn't match after '\n'

[done] python.exp (AIX) needs to have new symbols added

[fixed] README: use Digital Unix, not OSF/1.

[fixed] the vc40.mak and example.mak still has different linkage flags
(/MD vs. /MTd).  don't ask me what they're called in dev studio...

[done] remove TODO file from the distribution

[ignore] Lundh can't get ``CC=compiler ./configure'' to work

[fixed] 64-bit warnings in regexpr.c (Lundh)

[done] Set -Olimit 1500

[fixed] test_builtin is broken on 64-bit platforms:
    if hex(-16) != '0xfffffff0': raise TestFailed, 'hex(-16)'
alternative:
    if len(hex(-1)) != len(hex(sys.maxint)):
	raise TestFailed, 'len(hex(-1))'
    if hex(-16) not in ('0xfffffff0', '0xfffffffffffffff0'):
	raise TestFailed, 'hex(-16)'

[fixed] Donn Cave's problems with getpath.c
	- absolute VPATH doesn't do the right thing
	- stop after PREFIX

[fixed -- use O_NONBLOCK] Why doesn't FCNTL always define FNDELAY?

[fixed] re search and match were slowed down 10x by extraneous string copy

[fixed] 64-bit shift in cPickle.c:678

======================================================================

AMK's list:

Wishlist for Python 1.5:
========================

* Enhanced regex test suite (AMK)

* Updated cursesmodule (documented, supports ncurses)
	-- add conditionalized ncurses support and a way to detect it (OA)
	-- document the module (AMK)
	-- a few demos (Life game, text mode browser(?) ) (AMK & others)

* Update BSDDB module to work with version 2.0 of the DB code at http://www.bostic.com/db/ 
  (but 2.0 isn't compatible with 1.85--groan...)

* Did anything ever come of the DateTime class that someone had?

* More docstrings!

* Remove knowledge of the long int representation from Python/marshal.c 

* minor typo fixes 

* test suite for dumbdbm.py 

* whatswrong.py (suggested by Fredrik Lundh)

Documentation issues
====================

* Note from someone: calling __dict__ readonly is a massive piece of
misinformation, especially when the docs immediately show it being
modified. . .  (Technically incorrect--__dict__ returns a mutable
object, but you can't say i.__dict__={} )

* Document SocketServer.

* More documentation on threads

* Rework the tutorial, to avoid all the many "New features in Python
  1.X" sections (Guido)

* Updated Tkinter life preserver (Frederik Lundh)

* Better LaTeX typography--narrower margins, nicer appearance
	-- ragged setting & no hyphenation?
	-- Decrease the space after periods
	-- Try to remove footnotes
	-- Improve conversion to Texinfo and HTML

* Improve the index--look for references on making good book indices.
	GB: A global master index of all identifiers
	    and macros, with descriptions, references to point of
            definition, and scope, would kick some *serious* butt, but
            may require some serious sweat...

* Text at the start of the index encouraging the user to report index
  flaws to some e-mail address

* Crossreferences across modules.  For example, rand.py could have 
  "Related modules: whrandom (sect. X.Y.Z), random (sect. I.J.K)"

* A style guide for the documentation

* Use macros and styles consistently throughout -- this will involve
  writing a style guide, and then reading through the TeX source
  looking for nits... 

======================================================================
my own older lists...
======================================================================

Bugs:

... exception in __del__ when __builtins__ has been deleted

Features:

... simple threading support (with lock) for Tk

Documentation:

... Add section on filesystem interface to tutorial

... Update tutorial for changes in the language & library

... New sections for library reference manual (use NEWS file for hints)

... add stuff to ext.tex about threads and embedded Python

... mention that -2147483648 doesn't work

... mention that 0e0 and 01.0 etc. don't work

Maybe not (yet):

... get rid of 'owner' passed around for newframeobject and evalcode

... rename init<module> to PyInit_<module>.

... import ni seems to interpret the current directory as a package.

... prototype for PyOS_CheckStack() somewhere

... On some Linuxes, ``python -c "print 12" | wc'' produces no output???

... Add INSTALL_ROOT variable to Makefile.in

... More thread testing (Solaris?), wait for David Arnold

... Get rid of more modules: (stdwin, soundex, SGI specific?)

... Use autoconf features to find X11 libraries

... Use more modern autoconf features (e.g. for platform testing)

======================================================================
======================================================================
The following is old and I haven't checked whether it still applies:

(-) many module should export their symbolic constants instead of
relying on a module written in Python

(-) change regexmodule.c to cooperate with other non-python users and
to export the symbolic constants

(-) save/restore sys.exc_{type,value,traceback} around except clauses.

(-) don't call class instance's __del__ more than once?????

(-) add "access" to posix?  What name should it have?

(-) add facility to "freeze" lists and dictionaries?

(-) add WNOHANG to posix

(-) support lists in newgetargs()

(-) syntax errors detected during compilation should give line number

(-) dbm.open(): rwmode, filemode should be made optional; same for gdbm

(-) find a bsd hash interface

(-) posix.mkdir(): mode should be made optional

(-) find a more useful order than alphabetical for Doc/libfuncs.tex

======================================================================

(-) investigate PPRC <URL:ftp://ftp.parc.xerox.com/pub/ppcr/>

(-) interface to getdtablesize() in posix

(-) reentrancy with global variables vs. decref in
./Modules/cdmodule.c ./Modules/flmodule.c ./Objects/accessobject.c
./Objects/frameobject.c ./Python/traceback.c

(-) speed up regsub.gsub

(-) try posixenviron.c and merge back into posixmodule.c?

(-) add and document chroot() ?

(-) Add `@CFLAGS@', `@CPPFLAGS@', and `@LDFLAGS@' to `Makefile.in'

(-) modules should be able to define a module destructor hook

(-) destroy modules in reverse order of importation?

(-) make array a standard built-in object

(-) makesetup should accept .o files without corresponding .c file

(-) try Boehm/Dehmers/Weiser conservative garbage collector

(-) document new Python/C API

(-) add various things to module dictionary, e.g. pathname, dictionary
where found, __version__ string?

(-) pass dict of builtins to exec / execfile / eval ???

(?) stack frame correspondence problem (Jim Roskind)
(probably solved by err_fetch / err_restore)

(-) make lots of places use newgetargs

(-) readline 2.0 on sequent has ^C problem (works only first time)

(-) need to add truncate() and ftruncate() to posixmodule.c (Windows:
chsize(fd, size) (solution: added to file objects instead)

(-) improve performance of list.append/insert etc. by keeping high/low
watermark instead of realloc'ing each time?

(-) findmethod should cache (also findmember?)

(-) sysget("check_interval") is called before each method call which
breaks dictlookup caching

(-) core dump on repr / print of deeply nested or recursive object

(-) whrandom doc needs update

(-) fix signalmodule.c to re-establish SIGC[H]LD handler

(-) redesign error handling (cf. Donald's mail)

(-) does MPW 3.2 need the MPW_881_BUG defined in Parser/acceler.c

(-) rename MPW_3_1 define (which really means 3.x)

(-) add warning to docs about sys.exc_traceback and sys.last_traceback.

(-) latex docs for signal module

(-) systematically create /usr/local/lib/python/<machine>-<os>/
    subdirectories, with a lib/ subdirectory containing the lib*.a
    files etc.

(-) need newer DOS binary (16 bit version doesn't do default args)

(-) document Tk

(-) class browser

(-) interactive Python GUI (a la NT thingie)

(-) more stuff under CVS (demo, extensions)

(-) use const for char * parameters (and many more) where possible

(-) The Great Renaming!

(-) document __getattr__, __setattr__

(-) many things that take strings should also take arrays of chars

(-) add list of existing extensions to FAQ

(-) update "recent additions" chapter in tutorial

(-) rewrite "output formatting" chapter in tutorial

(-) document addpack, urllib, ...

(-) make regsub.[g]sub() optionally case insensitive

(-) dynamic linking on the Mac (is this a dream?)

(-) Later:
    - put the offending object in IOError and posix.error
    - make module marshal work with user-defined file-like objects
    - built-in help?
    - hierarchical module names?

Big plans:

- allow separate interpreters (a la Xt's Applocation Contexts, and Tcl)
- complete reflexive nature of the language, e.g. have interfaces et
create any kind of object

FAQ:

why don't list methods return self

Even older:

make .pyc files executable (how?)

thread status and improvements (lock stmt; signal/wait)

optional optimizations

pthread migration

multiple interpreter objects

persistency

new Dbhash.py, dbhash library

(-) dbm objects miss items(), values() methods

(-) answer q about coerce()

(-) describe() ?

(-) distribute demo2 with Holmes

(-) classes are too slow

(-) add += etc. ?

optimize tuple = tuple

allow (a, b) = [1, 2] and [1, 2] = (1, 2) ???

explain rules about == vs. 'is' for strings (* by others on the list)

rewrite section on formatting in tutorial

======================================================================
TODO-TOO list:

test for overflow when converting python long to float

lift restrictions on tuple or list in many cases

Should double-check all changes with docs!

(?) Interrupting output still sometimes doesn't call clearerr() properly

sometimes ghost errors when interrupting during debugging in
'continue' mode?

typing a comment to a primary prompt shouldn't issue a secondary prompt

readline: add hooks to recognize Python syntax and to expand Python
commands and names

should have absolute pathnames in function objects

in general check that all the exceptions are modernized and that the
messages aren't giving the same error twice (e.g., stdwinmodule.c!)

- check read/write allowed for file objects

- introduce macros to set/inspect errno for syscalls, to support things
  like getoserr()

======================================================================
DOS/Windows Python

(???) command line options?

(???) interrupts

(???) wrap