"Fred L. Drake, Jr." wrote:

M.-A. Lemburg writes:

...

Does this mean I can still splip in that minor patch to allow for attribute doc-strings in 2.0b1 provided I write up a short PEP really fast ;-) ?

Write a PEP if you like; I think I'd really like to look at this before you change any code, and I've not had a chance to read your messages about this yet. This is *awefully* late to be making a change that hasn't been substantially hashed out and reviewed, and I'm under the impression that this is pretty new (the past week or so).

FYI, I've attached the pre-PEP below (I also sent it to Barry for review). This PEP is indeed very new, but AFAIK it doesn't harm any existing code and also doesn't add much code complexity to achieve what it's doing (see the patch).

...

BTW, what the new standard on releasing ideas to dev public ? I know I'll have to write a PEP, but where should I put the patch ? Into the SF patch manager or on a separate page on the Internet ?

Patches should still go to the SF patch manager.

Here it is: http://sourceforge.net/patch/?func=detailpatch&patch_id=101264&group_id=5470 -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ PEP: 224 Title: Attribute Docstrings Version: $Revision: 1.0 $ Owner: mal@lemburg.com (Marc-Andre Lemburg) Python-Version: 2.0 Status: Draft Created: 23-Aug-2000 Type: Standards Track Introduction This PEP describes the "attribute docstring" proposal for Python 2.0. This PEP tracks the status and ownership of this feature. It contains a description of the feature and outlines changes necessary to support the feature. The CVS revision history of this file contains the definitive historical record. Rationale This PEP proposes a small addition to the way Python currently handles docstrings embedded in Python code. Until now, Python only handles the case of docstrings which appear directly after a class definition, a function definition or as first string literal in a module. The string literals are added to the objects in question under the __doc__ attribute and are from then on available for introspecition tools which can extract the contained information for help, debugging and documentation purposes. Docstrings appearing in other locations as the ones mentioned are simply ignored and don't result in any code generation. Here is an example: class C: " class C doc-string " a = 1 " attribute C.a doc-string (1)" b = 2 " attribute C.b doc-string (2)" The docstrings (1) and (2) are currently being ignored by the Python byte code compiler, but could obviously be put to good use for documenting the named assignments that preceed them. This PEP proposes to also make use of these cases by proposing semantics for adding their content to the objects in which they appear under new generated attribute names. The original idea behind this approach which also inspired the above example was to enable inline documentation of class attributes, which can currently only be documented in the class' docstring or using comments which are not available for introspection. Implementation Docstrings are handled by the byte code compiler as expressions. The current implementation special cases the few locations mentioned above to make use of these expressions, but otherwise ignores the strings completely. To enable use of these docstrings for documenting named assignments (which is the natural way of defining e.g. class attributes), the compiler will have to keep track of the last assigned name and then use this name to assign the content of the docstring to an attribute of the containing object by means of storing it in as a constant which is then added to the object's namespace during object construction time. In order to preserve features like inheritence and hiding of Python's special attributes (ones with leading and trailing double underscores), a special name mangling has to be applied which uniquely identifies the docstring as belonging to the name assignment and allows finding the docstring later on by inspecting the namespace. The following name mangling scheme achieves all of the above: __doc__<attributename>__ To keep track of the last assigned name, the byte code compiler stores this name in a variable of the compiling structure. This variable defaults to NULL. When it sees a docstring, it then checks the variable and uses the name as basis for the above name mangling to produce an implicit assignment of the docstring to the mangled name. It then resets the variable to NULL to avoid duplicate assignments. If the variable does not point to a name (i.e. is NULL), no assignments are made. These will continue to be ignored like before. All classical docstrings fall under this case, so no duplicate assignments are done. In the above example this would result in the following new class attributes to be created: C.__doc__a__ == " attribute C.a doc-string (1)" C.__doc__b__ == " attribute C.b doc-string (2)" A patch to the current CVS version of Python 2.0 which implements the above is available on SourceForge at [1]. Caveats of the Implementation Since the implementation does not reset the compiling structure variable when processing a non-expression, e.g. a function definition, the last assigned name remains active until either the next assignment or the next occurrence of a docstring. This can lead to cases where the docstring and assignment may be separated by other expressions: class C: "C doc string" b = 2 def x(self): "C.x doc string" y = 3 return 1 "b's doc string" Since the definition of method "x" currently does not reset the used assignment name variable, it is still valid when the compiler reaches Copyright This document has been placed in the Public Domain. References [1] http://sourceforge.net/patch/?func=detailpatch&patch_id=101264&group_id=5470  Local Variables: mode: indented-text indent-tabs-mode: nil End:

Tim Peters wrote:

[MAL]

...

Does this mean I can still splip in that minor patch to allow for attribute doc-strings in 2.0b1 provided I write up a short PEP really fast ;-) ?

2.0 went into feature freeze the Monday *before* this one! So, no. The "no new features after 2.0b1" refers mostly to the patches currently in Open and Accepted: if *they're* not checked in before 2.0b1 goes out, they don't get into 2.0 either.

Ah, ok. Pity I just started to do some heaviy doc-string extracting just last week... oh, well.

Ideas that were accepted by Guido for 2.0 before last Monday aren't part of the general "feature freeze". Any new feature proposed *since* then has been Postponed without second thought. Guido accepted several ideas before feature freeze that still haven't been checked in (in some cases, still not coded!), and just dealing with them has already caused a slip in the schedule. We simply can't afford to entertain new ideas too now (indeed, that's why "feature freeze" exists: focus).

For you in particular <wink>, how about dealing with Open patch 100899? It's been assigned to you for 5 weeks, and if you're not going to review it or kick /F in the butt, assign it to someone else.

AFAIK, Fredrik hasn't continued work on that patch and some important parts are still missing, e.g. the generator scripts and a description of how the whole thing works. It's not that important though, since the patch is a space optimization of what is already in Python 2.0 (and has been for quite a while now): the Unicode database. Perhaps I should simply post-pone the patch to 2.1 ?!

...

BTW, what the new standard on releasing ideas to dev public ? I know I'll have to write a PEP, but where should I put the patch ? Into the SF patch manager or on a separate page on the Internet ?

The PEP should be posted to both Python-Dev and comp.lang.python after its first stab is done. If you don't at least post a link to the patch in the SF Patch Manager, the patch doesn't officially exist. I personally prefer one-stop shopping, and SF is the Python Developer's Mall; but there's no rule about that yet (note that 100899's patch was apparently so big SF wouldn't accept it, so /F *had* to post just a URL to the Patch Manager).

I've just posted the PEP here, CCed it to Barry and uploaded the patch to SF. I'll post it to c.l.p tomorrow (don't know what that's good for though, since I don't read c.l.p anymore). -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

[Jeremy Hylton]

I would like to see some compression in the release, but agree that it is not an essential optimization. People have talked about it for a couple of months, and we haven't found someone to work on it because at various times pirx and /F said they were working on it.

If we don't hear from /F by tomorrow promising he will finish it before the beta release, let's postpone it.

There was an *awful* lot of whining about the size increase without this optimization, and the current situation violates the "no compiler warnings!" rule too (at least under MSVC 6). That means it's going to fail to compile at all on *some* feebler system. We said we'd put it in, so I'm afraid I think it falls on PythonLabs to finish it if /F can't.

Thomas Wouters wrote:

On Wed, Aug 23, 2000 at 02:32:20PM -0400, Tim Peters wrote:

...

[Jeremy Hylton]

...

I would like to see some compression in the release, but agree that it is not an essential optimization. People have talked about it for a couple of months, and we haven't found someone to work on it because at various times pirx and /F said they were working on it.

If we don't hear from /F by tomorrow promising he will finish it before the beta release, let's postpone it.

...

There was an *awful* lot of whining about the size increase without this optimization, and the current situation violates the "no compiler warnings!" rule too (at least under MSVC 6).

For the record, you can't compile unicodedatabase.c with g++ because of it's size: g++ complains that the switch is too large to compile. Under gcc it compiles, but only by trying really really hard, and I don't know how it performs under other versions of gcc (in particular more heavily optimizing ones -- might run into other limits in those situations.)

Are you sure this is still true with the latest CVS tree version ? I split the unicodedatabase.c static array into chunks of 4096 entries each -- that should really be managable by all compilers. But perhaps you are talking about the switch in unicodectype.c (there are no large switches in unicodedatabase.c) ? In that case, Jack Janssen has added a macro switch which breaks that switch in multiple parts too (see the top of that file). It should be no problem adding a few more platforms to the list of platforms which have this switch defined per default (currently Macs and MS Win64). I see no problem taking the load off of Fredrik an postponing the patch to 2.1. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

[Fredrik Lundh] [on patch 100899]

mal has reviewed the patch, and is waiting for an update from me.

Thanks! On that basis, I've reassigned the patch to you.

PS. the best way to get me to do something is to add a task to the task manager.

Yikes! I haven't looked at the thing since the day after I enabled it <wink> -- thanks for the clue.

I currently have three things on my slate:

17333 add os.popen2 support for Unix

Guido definitely wants this for 2.0, but there's no patch for it and no entry in PEP 200. Jeremy, please add it.

17334 add PyErr_Format to errors module 17335 add compressed unicode database

Those two are in Open patches, and both assigned to you.

if I missed something, let me know.

In your email (to Guido and me) from Monday, 31-July-2000,

so to summarize, Python 2.0 will support the following hex-escapes:

\xNN \uNNNN \UNNNNNNNN

where the last two are only supported in Unicode and SRE strings.

I'll provide patches later this week, once the next SRE release is wrapped up (later tonight, I hope).

This apparently fell through the cracks, and I finally remembered it last Friday, and added them to PEP 200 recently. Guido wants this in 2.0, and accepted them long before feature-freeze. I'm currently writing a PEP for the \x change (because it has a surreal chance of breaking old code). I haven't written any code for it. The new \U escape is too minor to need a PEP (according to me).

[/F, on "add os.popen2 support for Unix"]

to reduce my load somewhat, maybe someone who does Python 2.0 development on a Unix box could produce that patch?

Sounds like a more than reasonable idea to me; heck, AFAIK, until you mentioned you thought it was on your plate, we didn't think it was on *anyone's* plate. It simply "came up" on its own at the PythonLabs mtg yesterday (which I misidentified as "Monday" in an earlier post). Can we get a volunteer here? Here's /F's explanation:

anyway, fixing this is pretty straightforward:

1) move the class (etc) from popen2.py to os.py

2) modify the "if hasattr" stuff; change

# popen2.py if hasattr(os, "popen2"): def popen2(...): # compatbility code, using os.popen2 else: def popen2(...): # unix implementation

to

# popen2.py def popen2(...): # compatibility code

# os.py def popen2(...) # unix implementation, with the order of # the return values changed to (child_stdin, # child_stdout, child_stderr)

[on \x, \u and \U]

if someone else can do the popen2 stuff, I'll take care of this one!

It's a deal as far as I'm concerned. Thanks! I'll finish the \x PEP anyway, though, as it's already in progress. Jeremy, please update PEP 200 accordingly (after you volunteer to do the os.popen2 etc bit for Unix(tm) <wink>).

I wanted to confirm: Tim is channeling the release manager just fine. We are in feature freeze for 2.0. Jeremy