Mailman 3 September 2005 - Python-Dev

Active Objects in Python
by Bruce Eckel Oct. 12, 2005

Oct. 12, 2005

According to this list's welcome message, I should introduce myself. I'm Bruce Eckel, I write, consult and give seminars about computer programming, I use Python whenever I can get away with it, and I've spoken at Pycon a number of times. There are further URLs in my signature at the bottom. I'm joining this list because I was having a conversation with Guido about concurrency support in Python (in particular, using an actor/active-object approach) and he suggested I bring it over here. Here are the highlights of my most recent reply to him (Guido's remarks are marked by the '>' signs): > Agreed. IMO the pthreads-style solution doesn't work well no matter > *what* the programming model. Exactly. I think the problem may be that low-level threads people are one step behind (much like embedded systems programmers, which is where I began). They may be just now catching up to objects, but as far as concurrency goes their brains still think in terms of threads, so it seems natural to apply thread concepts to objects. But from an OO standpoint, pthread-style thinking is two steps backwards. You effectively throw open the innards of the object that you just spent time decoupling from the rest of your system, and the coupling is now unpredictable. It's the worst of all possible worlds. > I worked on Mobile Agents at CNRI from 1995 till 2000, but they were > very heavy-weight. Yet, I think that any solution that moves objects > between CPUs on a regular basis is going to suffer the same problem > that process switching in general gets you -- the move is so expensive > that it's hard to decide when it's justified. The examples I've seen haven't relied on mobility, so that's a question: how important is mobility really to an agent system? And could it be done in a trivial fashion in Python, by sending source code. And if it was really necessary sometimes, could it be something where the cost of mobility only occurred when moving an object? It's possible that either an argument can be made against mobility and/or some kind of trivial mobility system could be developed that would work when necessary. Possibly a Linda-like put-take system with pickles (off the top of my head). I don't know that much about mobility, but it does seem like mobile agents could be a powerful solution to the problem solved by enterprise messaging systems. > I believe you pointed me to Active Objects before, right? (Years ago, > maybe when you visited us at Zope.) I may have mentioned them in the past. > If multiple active objects can co-exist in the same *process*, but > nevertheless the language implementation prevents them from sharing > data except via channels, and in addition allows dynamic reallocation > of active objects across multiple CPUs, they just might be the ticket. > But it would require a complete implementation to prove it. Yes, defining an class as "active" would: 1) Install a worker thread and concurrent queue in each object of that class. 2) Automatically turn method calls into tasks and enqueue them 3) Prevent any other interaction other than enqueued messages At that point, the only time the GIL might be needed is to lock the queue at the point of putting ant taking objects. But there has been work done on lock-free data structures, which has been incorporated into Java 5 libraries, so it might even be possible to use a lock-free queue to do this and thus eliminate the need for the GIL at all. Since the enqueuing process serializes all requests to active objects, and since each message-task runs to completion before the next one starts, the problem of threads interfering with each other is eliminated. Also, the enqueuing process will always happen, so even if the queue blocks it will be for a very short, deterministic time. So the theory is that Active Object systems cannot deadlock (although I believe they can livelock). So yes, the best way for this to work might be some kind of enforced implementation -- but forcing you to do something is not particularly Pythonic, so I would think that it would be possible to build an active object framework along with some checking tools to warn you if you are breaking the rules. But what's not clear is whether this would require knowledge of the innards of the Python interpreter (which I don't have) or if it could be built using libraries. BTW: I think that Hoare's Communicating Sequential Processes (CSP) basically describes the idea of active objects but without objects. That is, I think active objects is CSP applied to OO. Bruce Eckel http://www.BruceEckel.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel

16 24

C API doc fix
by Robey Pointer Oct. 10, 2005

Oct. 10, 2005

Yesterday I ran into a bug in the C API docs. The top of this page: http://docs.python.org/api/unicodeObjects.html says: Py_UNICODE This type represents a 16-bit unsigned storage type which is used by Python internally as basis for holding Unicode ordinals. On platforms where wchar_t is available and also has 16-bits, Py_UNICODE is a typedef alias for wchar_t to enhance native platform compatibility. On all other platforms, Py_UNICODE is a typedef alias for unsigned short. This is incorrect on some platforms: on Debian, Py_UNICODE turns out to be 32 bits. I'm not sure what the correct quote should be: Does python use wchar_t whenever it's available (16 bits or not)? I solved my problem by realizing that I was going about things entirely wrong, and that I should use the python codecs from C and not worry about what Py_UNICODE contains. However, I think we should fix the docs to avoid confusing others... or maybe it would be better to document what's in Py_UNICODE and suggest always using the codec methods? I don't have a strong opinion either way. robey

5 8

RFC: readproperty
by Jim Fulton Oct. 10, 2005

Oct. 10, 2005

Read descriptors (defining only __get__) and data descripttors have (defining __get__ and __set__) different semantics. In particular, read descriptors are overridden by instance data, while data descriptors are not. A common use of read descriptors is for lazily computed data: class readproperty(object): "Create a read descriptor from a function" def __init__(self, func): self.func = func def __get__(self, inst, class_): if inst is None: return self return self.func(inst) class Spam: @readproperty def eggs(self): ... expensive computation of eggs self.eggs = result return result When we ask for the eggs attribute the first time, we call the descriptor, which calls the eggs function. The function sets the eggs attribute, overriding the descriptor. The next time the eggs attribute is accessed, the saved value will be used without calling the descriptor. I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5. Thoughts? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

10 19

C API doc fix
by Jim Jewett Oct. 2, 2005

Oct. 2, 2005

On 9/29/05, Robey Pointer <robey at lag.net> wrote: > Yesterday I ran into a bug in the C API docs. The top of this page: > > http://docs.python.org/api/unicodeObjects.html > > says: > > Py_UNICODE > This type represents a 16-bit unsigned storage type which is > used by Python internally as basis for holding Unicode ordinals. On > platforms where wchar_t is available and also has 16-bits, Py_UNICODE > is a typedef alias for wchar_t to enhance native platform > compatibility. On all other platforms, Py_UNICODE is a typedef alias > for unsigned short. Steven Bethard wrote: > I believe this is the same issue that was brought up in May[1]. My > impression was that people could not agree on a documentation patch. > [1] http://www.python.org/dev/summary/2005-05-01_2005-05-15.html I thought the problem was disagreement over how the system *should* pick an underlying type to alias. Given the current policy, are there objections to a patch that at least steers people away from assuming they can use the underlying type directly? ======== Py_UNICODE Python uses this type to store Unicode ordinals. It is typically a typedef alias, but the underlying type -- and the size of that type -- varies across different systems. ======== -jJ

2 1

IDLE development
by Noam Raphael Oct. 2, 2005

Oct. 2, 2005

Hello, More than a year and a half ago, I posted a big patch to IDLE which adds support for completion and much better calltips, along with some other improvements. Since then, I had some mail conversations with Kurt B. Kaiser, who is responsible for IDLE, which resulted in nothing. My last mail, from Jul 10, saying (with more details) "I made the minor changes you asked for, let's get it in, it's not very complicated" was unanswered. This is just an example of the fact that IDLE development was virtually nonexistent in the last months, because most patches were simply ignored. I and my colleges use IDLE intensively - that is, a heavily patched IDLE. It includes my patch and many other improvements made by me and my friends. The improved IDLE is MUCH better than the standard IDLE, especially for interactive work. Since we would like to share our work with the rest of the world, if nothing is changed we would start a new IDLE fork soon, perhaps at python-hosting.com. I really don't like that - maintaining a fork requires a lot of extra work, and it is certain that many more people will enjoy our work if it integrated in the standard Python distribution. But sending patches and watching them stay open despite a continuous nagging is worse. Please, either convince KBK to invest more time in IDLE development, or find someone else who would take care of it. If you like, I would happily help in the development. I hope I am not sounding offensive. It's actually quite simple: if the excellent development environment IDLE can't develop inside standard Python, it should be developed outside it. As I said, I prefer the first option. Have a good week, Noam Raphael

4 5

Pythonic concurrency
by Jim Jewett Oct. 2, 2005

Oct. 2, 2005

Bruce Eckel wrote: > 3) Tasks are cheap enough that I can make > thousands of them, ... > 4) Tasks are "self-guarding," so they prevent > other tasks from interfering with them. The > only way tasks can communicate with each > other is through some kind of formal > mechanism (something queue-ish, > I'd imagine). I think these two are the hardest to reconcile. Shane Hathaway's suggestion starts from the process end. A new process isn't cheap. Keeping the other process' interpreter alive and feeding it more requests through a queue just hides the problem; you can't have more non-sequential tasks than processors without restarting the whole contention issue. Even using sequential tasks (similar to "import dummy_thread") lets task1 mess up the builtins (or other library modules) for future task2. The more guards you add, the heavier each task gets. At the other end are generators; I think what generators are missing is (A) You can't easily send them a message. This can be solved by wrapping them in an object, or (probably) by waiting until 2.5. (B) The programmer has to supply a scheduler. This could be solved by a standard library module. (C) That scheduler is non-preemptive. A single greedy generator can starve all the others. You can reduce the problem by scheduling generators on more than one thread. To really solve it would require language support. That *might* be almost as simple as a new object type that automatically yielded control every so often (like threads). (D) Generators can interfere with each other unless the programmer is careful to treat all externally visible objects as immutable. Again, it would require language support. I also have a vague feeling that fixing (C) or (D) might make the other worse. (E) Generators are not reentrant. I know you said that some restrictions are reasonable, and this might fall into that category ... but I think this becomes a problem as soon as Tasks can handle more than one type of message, or can send delayed replies to specific correspondents. (To finish your request, I need some more information...) -jJ

14 18

PEP 350: Codetags
by Micah Elliott Oct. 1, 2005

Oct. 1, 2005

Please read/comment/vote. This circulated as a pre-PEP proposal submitted to c.l.py on August 10, but has changed quite a bit since then. I'm reposting this since it is now "Open (under consideration)" at <http://www.python.org/peps/pep-0350.html>. Thanks! -- Micah Elliott <mde at tracos.org> PEP: 350 Title: Codetags Version: $Revision: 1.2 $ Last-Modified: $Date: 2005/09/26 19:56:53 $ Author: Micah Elliott <mde at tracos.org> Status: Draft Type: Informational Content-Type: text/x-rst Created: 27-Jun-2005 Post-History: 10-Aug-2005, 26-Sep-2005 Abstract ======== This informational PEP aims to provide guidelines for consistent use of *codetags*, which would enable the construction of standard utilities to take advantage of the codetag information, as well as making Python code more uniform across projects. Codetags also represent a very lightweight programming micro-paradigm and become useful for project management, documentation, change tracking, and project health monitoring. This is submitted as a PEP because its ideas are thought to be Pythonic, although the concepts are not unique to Python programming. Herein are the definition of codetags, the philosophy behind them, a motivation for standardized conventions, some examples, a specification, a toolset description, and possible objections to the Codetag project/paradigm. This PEP is also living as a wiki_ for people to add comments. What Are Codetags? ================== Programmers widely use ad-hoc code comment markup conventions to serve as reminders of sections of code that need closer inspection or review. Examples of markup include ``FIXME``, ``TODO``, ``XXX``, ``BUG``, but there many more in wide use in existing software. Such markup will henceforth be referred to as *codetags*. These codetags may show up in application code, unit tests, scripts, general documentation, or wherever suitable. Codetags have been under discussion and in use (hundreds of codetags in the Python 2.4 sources) in many places (e.g., c2_) for many years. See References_ for further historic and current information. Philosophy ========== If you subscribe to most of these values, then codetags will likely be useful for you. 1. As much information as possible should be contained **inside the source code** (application code or unit tests). This along with use of codetags impedes duplication. Most documentation can be generated from that source code; e.g., by using help2man, man2html, docutils, epydoc/pydoc, ctdoc, etc. 2. Information should be almost **never duplicated** -- it should be recorded in a single original format and all other locations should be automatically generated from the original, or simply be referenced. This is famously known as the Single Point Of Truth (SPOT) or Don't Repeat Yourself (DRY) rule. 3. Documentation that gets into customers' hands should be **auto-generated** from single sources into all other output formats. People want documentation in many forms. It is thus important to have a documentation system that can generate all of these. 4. The **developers are the documentation team**. They write the code and should know the code the best. There should not be a dedicated, disjoint documentation team for any non-huge project. 5. **Plain text** (with non-invasive markup) is the best format for writing anything. All other formats are to be generated from the plain text. Codetag design was influenced by the following goals: A. Comments should be short whenever possible. B. Codetag fields should be optional and of minimal length. Default values and custom fields can be set by individual code shops. C. Codetags should be minimalistic. The quicker it is to jot something down, the more likely it is to get jotted. D. The most common use of codetags will only have zero to two fields specified, and these should be the easiest to type and read. Motivation ========== * **Various productivity tools can be built around codetags.** See Tools_. * **Encourages consistency.** Historically, a subset of these codetags has been used informally in the majority of source code in existence, whether in Python or in other languages. Tags have been used in an inconsistent manner with different spellings, semantics, format, and placement. For example, some programmers might include datestamps and/or user identifiers, limit to a single line or not, spell the codetag differently than others, etc. * **Encourages adherence to SPOT/DRY principle.** E.g., generating a roadmap dynamically from codetags instead of keeping TODOs in sync with separate roadmap document. * **Easy to remember.** All codetags must be concise, intuitive, and semantically non-overlapping with others. The format must also be simple. * **Use not required/imposed.** If you don't use codetags already, there's no obligation to start, and no risk of affecting code (but see Objections_). A small subset can be adopted and the Tools_ will still be useful (a few codetags have probably already been adopted on an ad-hoc basis anyway). Also it is very easy to identify and remove (and possibly record) a codetag that is no longer deemed useful. * **Gives a global view of code.** Tools can be used to generate documentation and reports. * **A logical location for capturing CRCs/Stories/Requirements.** The XP community often does not electronically capture Stories, but codetags seem like a good place to locate them. * **Extremely lightweight process.** Creating tickets in a tracking system for every thought degrades development velocity. Even if a ticketing system is employed, codetags are useful for simply containing links to those tickets. Examples ======== This shows a simple codetag as commonly found in sources everywhere (with the addition of a trailing ``<>``):: # FIXME: Seems like this loop should be finite. <> while True: ... The following contrived example demonstrates a typical use of codetags. It uses some of the available fields to specify the assignees (a pair of programmers with initials *MDE* and *CLE*), the Date of expected completion (*Week 14*), and the Priority of the item (*2*):: # FIXME: Seems like this loop should be finite. <MDE,CLE d:14w p:2> while True: ... This codetag shows a bug with fields describing author, discovery (origination) date, due date, and priority:: # BUG: Crashes if run on Sundays. # <MDE 2005-09-04 d:14w p:2> if day == 'Sunday': ... Here is a demonstration of how not to use codetags. This has many problems: 1) Codetags cannot share a line with code; 2) Missing colon after mnemonic; 3) A codetag referring to codetags is usually useless, and worse, it is not completable; 4) No need to have a bunch of fields for a trivial codetag; 5) Fields with unknown values (``t:XXX``) should not be used:: i = i + 1 # TODO Add some more codetags. # <JRNewbie 2005-04-03 d:2005-09-03 t:XXX d:14w p:0 s:inprogress> Specification ============= This describes the format: syntax, mnemonic names, fields, and semantics, and also the separate DONE File. General Syntax -------------- Each codetag should be inside a comment, and can be any number of lines. It should not share a line with code. It should match the indentation of surrounding code. The end of the codetag is marked by a pair of angle brackets ``<>`` containing optional fields, which must not be split onto multiple lines. It is preferred to have a codetag in ``#`` comments instead of string comments. There can be multiple fields per codetag, all of which are optional. .. NOTE: It may be reasonable to allow fields to fit on multiple lines, but it complicates parsing and defeats minimalism if you use this many fields. In short, a codetag consists of a mnemonic, a colon, commentary text, an opening angle bracket, an optional list of fields, and a closing angle bracket. E.g., :: # MNEMONIC: Some (maybe multi-line) commentary. <field field ...> Mnemonics --------- The codetags of interest are listed below, using the following format: | ``recommended mnemonic (& synonym list)`` | *canonical name*: semantics ``TODO (MILESTONE, MLSTN, DONE, YAGNI, TBD, TOBEDONE)`` *To do*: Informal tasks/features that are pending completion. ``FIXME (XXX, DEBUG, BROKEN, REFACTOR, REFACT, RFCTR, OOPS, SMELL, NEEDSWORK, INSPECT)`` *Fix me*: Areas of problematic or ugly code needing refactoring or cleanup. ``BUG (BUGFIX)`` *Bugs*: Reported defects tracked in bug database. ``NOBUG (NOFIX, WONTFIX, DONTFIX, NEVERFIX, UNFIXABLE, CANTFIX)`` *Will Not Be Fixed*: Problems that are well-known but will never be addressed due to design problems or domain limitations. ``REQ (REQUIREMENT, STORY)`` *Requirements*: Satisfactions of specific, formal requirements. ``RFE (FEETCH, NYI, FR, FTRQ, FTR)`` *Requests For Enhancement*: Roadmap items not yet implemented. ``IDEA`` *Ideas*: Possible RFE candidates, but less formal than RFE. ``??? (QUESTION, QUEST, QSTN, WTF)`` *Questions*: Misunderstood details. ``!!! (ALERT)`` *Alerts*: In need of immediate attention. ``HACK (CLEVER, MAGIC)`` *Hacks*: Temporary code to force inflexible functionality, or simply a test change, or workaround a known problem. ``PORT (PORTABILITY, WKRD)`` *Portability*: Workarounds specific to OS, Python version, etc. ``CAVEAT (CAV, CAVT, WARNING, CAUTION)`` *Caveats*: Implementation details/gotchas that stand out as non-intuitive. ``NOTE (HELP)`` *Notes*: Sections where a code reviewer found something that needs discussion or further investigation. ``FAQ`` *Frequently Asked Questions*: Interesting areas that require external explanation. ``GLOSS (GLOSSARY)`` *Glossary*: Definitions for project glossary. ``SEE (REF, REFERENCE)`` *See*: Pointers to other code, web link, etc. ``TODOC (DOCDO, DODOC, NEEDSDOC, EXPLAIN, DOCUMENT)`` *Needs Documentation*: Areas of code that still need to be documented. ``CRED (CREDIT, THANKS)`` *Credits*: Accreditations for external provision of enlightenment. ``STAT (STATUS)`` *Status*: File-level statistical indicator of maturity of this file. ``RVD (REVIEWED, REVIEW)`` *Reviewed*: File-level indicator that review was conducted. File-level codetags might be better suited as properties in the revision control system, but might still be appropriately specified in a codetag. Some of these are temporary (e.g., ``FIXME``) while others are persistent (e.g., ``REQ``). A mnemonic was chosen over a synonym using three criteria: descriptiveness, length (shorter is better), commonly used. Choosing between ``FIXME`` and ``XXX`` is difficult. ``XXX`` seems to be more common, but much less descriptive. Furthermore, ``XXX`` is a useful placeholder in a piece of code having a value that is unknown. Thus ``FIXME`` is the preferred spelling. `Sun says`__ that ``XXX`` and ``FIXME`` are slightly different, giving ``XXX`` higher severity. However, with decades of chaos on this topic, and too many millions of developers who won't be influenced by Sun, it is easy to rightly call them synonyms. __ http://java.sun.com/docs/codeconv/html/CodeConventions.doc9.html#395 ``DONE`` is always a completed ``TODO`` item, but this should probably be indicated through the revision control system and/or a completion recording mechanism (see `DONE File`_). It may be a useful metric to count ``NOTE`` tags: a high count may indicate a design (or other) problem. But of course the majority of codetags indicate areas of code needing some attention. An ``FAQ`` is probably more appropriately documented in a wiki where users can more easily view and contribute. Fields ------ All fields are optional. The proposed standard fields are described in this section. Note that upper case field characters are intended to be replaced. The *Originator/Assignee* and *Origination Date/Week* fields are the most common and don't usually require a prefix. .. NOTE: the colon after the prefix is a new addition that became necessary when it was pointed out that a "codename" field (with no digits) such as "cTiger" would be indistinguishable from a username. <MDE 2005-9-24> .. NOTE: This section started out with just assignee and due week. It has grown into a lot of fields by request. It is still probably best to use a tracking system for any items that deserve it, and not duplicate everything in a codetag (field). <MDE> This lengthy list of fields is liable to scare people (the intended minimalists) away from adopting codetags, but keep in mind that these only exist to support programmers who either 1) like to keep ``BUG`` or ``RFE`` codetags in a complete form, or 2) are using codetags as their complete and only tracking system. In other words, many of these fields will be used very rarely. They are gathered largely from industry-wide conventions, and example sources include `GCC Bugzilla`__ and `Python's SourceForge`__ tracking systems. .. ???: Maybe codetags inside packages (__init__.py files) could have special global significance. <MDE> __ http://gcc.gnu.org/bugzilla/ __ http://sourceforge.net/tracker/?group_id=5470 ``AAA[,BBB]...`` List of *Originator* or *Assignee* initials (the context determines which unless both should exist). It is also okay to use usernames such as ``MicahE`` instead of initials. Initials (in upper case) are the preferred form. ``a:AAA[,BBB]...`` List of *Assignee* initials. This is necessary only in (rare) cases where a codetag has both an assignee and an originator, and they are different. Otherwise the ``a:`` prefix is omitted, and context determines the intent. E.g., ``FIXME`` usually has an *Assignee*, and ``NOTE`` usually has an *Originator*, but if a ``FIXME`` was originated (and initialed) by a reviewer, then the assignee's initials would need a ``a:`` prefix. ``YYYY[-MM[-DD]]`` or ``WW[.D]w`` The *Origination Date* indicating when the comment was added, in `ISO 8601`_ format (digits and hyphens only). Or *Origination Week*, an alternative form for specifying an *Origination Date*. A day of the week can be optionally specified. The ``w`` suffix is necessary for distinguishing from a date. ``d:YYYY[-MM[-DD]]`` or ``d:WW[.D]w`` *Due Date (d)* target completion (estimate). Or *Due Week (d)*, an alternative to specifying a *Due Date*. ``p:N`` *Priority (p)* level. Range (N) is from 0..3 with 3 being the highest. 0..3 are analogous to low, medium, high, and showstopper/critical. The *Severity* field could be factored into this single number, and doing so is recommended since having both is subject to varying interpretation. The range and order should be customizable. The existence of this field is important for any tool that itemizes codetags. Thus a (customizable) default value should be supported. ``t:NNNN`` *Tracker (t)* number corresponding to associated Ticket ID in separate tracking system. The following fields are also available but expected to be less common. ``c:AAAA`` *Category (c)* indicating some specific area affected by this item. ``s:AAAA`` *Status (s)* indicating state of item. Examples are "unexplored", "understood", "inprogress", "fixed", "done", "closed". Note that when an item is completed it is probably better to remove the codetag and record it in a `DONE File`_. ``i:N`` Development cycle *Iteration (i)*. Useful for grouping codetags into completion target groups. ``r:N`` Development cycle *Release (r)*. Useful for grouping codetags into completion target groups. .. NOTE: SourceForge does not recognize a severity and I think that *Priority* (along with separate RFE codetags) should encompass and obviate *Severity*. <MDE> .. NOTE: The tools will need an ability to sort codetags in order of targeted completion. I feel that *Priority* should be a unique, lone indicator of that addressability order. Other categories such as *Severity*, *Customer Importance*, etc. are related to business logic and should not be recognized by the codetag tools. If some groups want to have such logic, then it is best factored (externally) into a single value (priority) that can determine an ordering of actionable items. <MDE> To summarize, the non-prefixed fields are initials and origination date, and the prefixed fields are: assignee (a), due (d), priority (p),tracker (t), category (c), status (s), iteration (i), and release (r). It should be possible for groups to define or add their own fields, and these should have upper case prefixes to distinguish them from the standard set. Examples of custom fields are *Operating System (O)*, *Severity (S)*, *Affected Version (A)*, *Customer (C)*, etc. DONE File --------- Some codetags have an ability to be *completed* (e.g., ``FIXME``, ``TODO``, ``BUG``). It is often important to retain completed items by recording them with a completion date stamp. Such completed items are best stored in a single location, global to a project (or maybe a package). The proposed format is most easily described by an example, say ``~/src/fooproj/DONE``:: # TODO: Recurse into subdirs only on blue # moons. <MDE 2003-09-26> [2005-09-26 Oops, I underestimated this one a bit. Should have used Warsaw's First Law!] # FIXME: ... ... You can see that the codetag is copied verbatim from the original source file. The date stamp is then entered on the following line with an optional post-mortem commentary. The entry is terminated by a blank line (``\n\n``). It may sound burdensome to have to delete codetag lines every time one gets completed. But in practice it is quite easy to setup a Vim or Emacs mapping to auto-record a codetag deletion in this format (sans the commentary). Tools ===== Currently, programmers (and sometimes analysts) typically use *grep* to generate a list of items corresponding to a single codetag. However, various hypothetical productivity tools could take advantage of a consistent codetag format. Some example tools follow. .. NOTE: Codetag tools are mostly unimplemented (but I'm getting started!) <MDE> Document Generator Possible docs: glossary, roadmap, manpages Codetag History Track (with revision control system interface) when a ``BUG`` tag (or any codetag) originated/resolved in a code section Code Statistics A project Health-O-Meter Codetag Lint Notify of invalid use of codetags, and aid in porting to codetags Story Manager/Browser An electronic means to replace XP notecards. In MVC terms, the codetag is the Model, and the Story Manager could be a graphical Viewer/Controller to do visual rearrangement, prioritization, and assignment, milestone management. Any Text Editor Used for changing, removing, adding, rearranging, recording codetags. There are some tools already in existence that take advantage of a smaller set of pseudo-codetags (see References_). There is also an example codetags implementation under way, known as the `Codetag Project`__. __ http://tracos.org/codetag Objections ========== :Objection: Extreme Programming argues that such codetags should not ever exist in code since the code is the documentation. :Defense: Maybe you should put the codetags in the unit test files instead. Besides, it's tough to generate documentation from uncommented source code. ---- :Objection: Too much existing code has not followed proposed guidelines. :Defense: [Simple] utilities (*ctlint*) could convert existing codes. ---- :Objection: Causes duplication with tracking system. :Defense: Not really, unless fields are abused. If an item exists in the tracker, a simple ticket number in the codetag tracker field is sufficient. Maybe a duplicated title would be acceptable. Furthermore, it's too burdensome to have a ticket filed for every item that pops into a developer's mind on-the-go. Additionally, the tracking system could possibly be obviated for simple or small projects that can reasonably fit the relevant data into a codetag. ---- :Objection: Codetags are ugly and clutter code. :Defense: That is a good point. But I'd still rather have such info in a single place (the source code) than various other documents, likely getting duplicated or forgotten about. The completed codetags can be sent off to the `DONE File`_, or to the bit bucket. ---- :Objection: Codetags (and allcomments) get out of date. :Defense: Not so much if other sources (externally visible documentation) depend on their being accurate. ---- :Objection: Codetags tend to only rarely have estimated completion dates of any sort. OK, the fields are optional, but you want to suggest fields that actually will be widely used. :Defense: If an item is inestimable don't bother with specifying a date field. Using tools to display items with order and/or color by due date and/or priority, it is easier to make estimates. Having your roadmap be a dynamic reflection of your codetags makes you much more likely to keep the codetags accurate. ---- :Objection: Named variables for the field parameters in the ``<>`` should be used instead of cryptic one-character prefixes. I.e., <MDE p:3> should rather be <author=MDE, priority=3>. :Defense: It is just too much typing/verbosity to spell out fields. I argue that ``p:3 i:2`` is as readable as ``priority=3, iteration=2`` and is much more likely to by typed and remembered (see bullet C in Philosophy_). In this case practicality beats purity. There are not many fields to keep track of so one letter prefixes are suitable. ---- :Objection: Synonyms should be deprecated since it is better to have a single way to spell something. :Defense: Many programmers prefer short mnemonic names, especially in comments. This is why short mnemonics were chosen as the primary names. However, others feel that an explicit spelling is less confusing and less prone to error. There will always be two camps on this subject. Thus synonyms (and complete, full spellings) should remain supported. ---- :Objection: It is cruel to use [for mnemonics] opaque acronyms and abbreviations which drop vowels; it's hard to figure these things out. On that basis I hate: MLSTN RFCTR RFE FEETCH, NYI, FR, FTRQ, FTR WKRD RVDBY :Defense: Mnemonics are preferred since they are pretty easy to remember and take up less space. If programmers didn't like dropping vowels we would be able to fit very little code on a line. The space is important for those who write comments that often fit on a single line. But when using a canon everywhere it is much less likely to get something to fit on a line. ---- :Objection: It takes too long to type the fields. :Defense: Then don't use (most or any of) them, especially if you're the only programmer. Terminating a codetag with ``<>`` is a small chore, and in doing so you enable the use of the proposed tools. Editor auto-completion of codetags is also useful: You can program your editor to stamp a template (e.g. ``# FIXME . <MDE {date}>``) with just a keystroke or two. ---- :Objection: *WorkWeek* is an obscure and uncommon time unit. :Defense: That's true but it is a highly suitable unit of granularity for estimation/targeting purposes, and it is very compact. The `ISO 8601`_ is widely understood but allows you to only specify either a specific day (restrictive) or month (broad). ---- :Objection: I aesthetically dislike for the comment to be terminated with <> in the empty field case. :Defense: It is necessary to have a terminator since codetags may be followed by non-codetag comments. Or codetags could be limited to a single line, but that's prohibitive. I can't think of any single-character terminator that is appropriate and significantly better than <>. Maybe ``@`` could be a terminator, but then most codetags will have an unnecessary @. ---- :Objection: I can't use codetags when writing HTML, or less specifically, XML. Maybe ``@fields@`` would be a better than ``<fields>`` as the delimiters. :Defense: Maybe you're right, but ``<>`` looks nicer whenever applicable. XML/SGML could use ``@`` while more common programming languages stick to ``<>``. References ========== Some other tools have approached defining/exploiting codetags. See http://tracos.org/codetag/wiki/Links. .. _wiki: http://tracos.org/codetag/wiki/Pep .. _ISO 8601: http://en.wikipedia.org/wiki/ISO_8601 .. _c2: http://c2.com/cgi/wiki?FixmeComment .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

11 15

Conditional Expression Resolution
by Guido van Rossum Sept. 30, 2005

Sept. 30, 2005

After a long discussion I've decided to add a shortcut conditional expression to Python 2.5. The syntax will be A if C else B This first evaluates C; if it is true, A is evaluated to give the result, otherwise, B is evaluated to give the result. The priorities will be such that you can write x = A if C else B x = lambda: A if C else B x = A if C else B if D else E But you'd have to write if (A if C else B): [x for x in seq if (A if C else B)] A if (X if C else Y) else B (A if C else B) if D else E Note that all these are intentionally ugly. :) In general, 'if' and 'else' bind less tight than everything except lambda. We will adjust the syntax of what goes inside an 'if' to disallow lambda; currently if lambda: x: is accepted but quite useless (it's always true) so this will be disallowed. Flames, pleas to reconsider, etc., to /dev/null. Congratulations gracefully accepted. It's still my language! :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

11 10

bool(container) [was bool(iter([])) changed between 2.3 and 2.4]
by Jim Jewett Sept. 30, 2005

Sept. 30, 2005

Python doesn't worry about a precise boolean object, it distinguishes between something and nothing. Is there anything left?" is a pretty good analogy for iterators. It isn't always cheaply available, and having might encourage poor style -- so iterators are going back to the default for non-containers of always True. How general strong is this default-to-true rule? If I submit a documentation patch, should I say that numbers, lists, strings, dictionaries, and tuples are a special case, or should I just warn that some container-like objects (including iterators) are always True? More specific question: A Queue.Queue is always true. Should I submit a bug patch or add it as a beware of custom classes example? def __nonzero__(self): return not self.empty()

5 4

Re: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4]
by Michael Chermside Sept. 30, 2005

Sept. 30, 2005

Jim Jewett writes: > Python doesn't worry about a precise boolean object, it > distinguishes between something and nothing. > > Is there anything left?" is a pretty good analogy for iterators. [...] > A Queue.Queue is always true. Should I submit a bug patch I would have phrased this very differently. I would have said it as follows: All Python objects can be converted to a boolean value. Most objects are True, except for False (of course), None, any number equal to 0, an empty string (""), and the empty containers (), [], and {}. User defined objects can override __len__ or __nonzero__ to return False; everything else is True. I don't believe that there is an overarching rule that all "containers" must be false when empty. I'm not sure there is even a clear definition of "container" that would satisfy everyone. It makes complete sense to me that empty iterators and Queue.Queues are True... it follows the general rule that everything is true except the above (short) list of objects and user-defined (or library) classes that want to mimic the behavior of one of these or have some obvious meaning for non-true. -- Michael Chermside

1 0