From fdrake@acm.org Mon Apr 1 16:37:06 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Mon, 1 Apr 2002 11:37:06 -0500 (EST)
Subject: [Doc-SIG] [development doc updates]
Message-ID: <20020401163706.D36FF18ED64@grendel.zope.com>
The development version of the documentation has been updated:
http://python.sourceforge.net/maint-docs/
Documentation being prepared for Python 2.2.1. This version includes
Andrew Kuchling's "What's New in Python 2.2" alongside the original
documents.
From fdrake@acm.org Mon Apr 1 20:33:00 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 1 Apr 2002 15:33:00 -0500
Subject: [Doc-SIG] New home for development version of docs
Message-ID: <15528.50172.390695.897521@grendel.zope.com>
The "in-developement" version of the documentation, previously found
at python.sourceforge.net, is now available from the development area
on python.org:
http://www.python.org/dev/doc/
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From fdrake@acm.org Mon Apr 1 22:06:24 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Mon, 1 Apr 2002 17:06:24 -0500 (EST)
Subject: [Doc-SIG] [development doc updates]
Message-ID: <20020401220624.423C818EAD1@grendel.zope.com>
The development version of the documentation has been updated:
http://www.python.org/dev/doc/devel/
Updated documentation on lexical scoping in the language reference.
From fdrake@acm.org Wed Apr 3 22:54:42 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Wed, 3 Apr 2002 17:54:42 -0500 (EST)
Subject: [Doc-SIG] [development doc updates]
Message-ID: <20020403225442.0343418EAD1@grendel.zope.com>
The development version of the documentation has been updated:
http://www.python.org/dev/doc/devel/
Updated to include whatever Guido did about bool().
From fdrake@acm.org Thu Apr 4 04:32:19 2002
From: fdrake@acm.org (Fred L. Drake)
Date: Wed, 3 Apr 2002 23:32:19 -0500 (EST)
Subject: [Doc-SIG] [maintenance doc updates]
Message-ID: <20020404043219.912DC28696@beowolf.fdrake.net>
The maintenance version of the documentation has been updated:
http://python.sourceforge.net/maint21-docs/
Preliminary documentation for Python 2.1.3.
From jafo-python-dev@tummy.com Fri Apr 5 00:47:52 2002
From: jafo-python-dev@tummy.com (Sean Reifschneider)
Date: Thu, 4 Apr 2002 17:47:52 -0700
Subject: [Doc-SIG] Re: [Python-Dev] [development doc updates]
In-Reply-To: <20020404225927.530B318EAD1@grendel.zope.com>; from fdrake@acm.org on Thu, Apr 04, 2002 at 05:59:27PM -0500
References: <20020404225927.530B318EAD1@grendel.zope.com>
Message-ID: <20020404174752.M16962@tummy.com>
On Thu, Apr 04, 2002 at 05:59:27PM -0500, Fred L. Drake wrote:
>The development version of the documentation has been updated:
On the topic of dev documentation... What happened to the info
distribution? I had info and HTML in the 2.2 RPM, but only the HTML were
around for 2.2.1c2 when I tried to pick it up the other night... Is there
going to be an info release for the 2.2.1 final, or should I just leave it
as HTML-only?
Sean
--
Tragedy is when I cut my finger. Comedy is when you fall into an open
sewer and die. -- Mel Brooks
Sean Reifschneider, Inimitably Superfluous
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
From fdrake@acm.org Fri Apr 5 01:07:31 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 4 Apr 2002 20:07:31 -0500
Subject: [Doc-SIG] Re: [Python-Dev] [development doc updates]
In-Reply-To: <20020404174752.M16962@tummy.com>
References: <20020404225927.530B318EAD1@grendel.zope.com>
<20020404174752.M16962@tummy.com>
Message-ID: <15532.63699.122156.945015@grendel.zope.com>
[Let's take this to the Doc-SIG; all followups there, please!]
Sean Reifschneider writes:
> On the topic of dev documentation... What happened to the info
> distribution? I had info and HTML in the 2.2 RPM, but only the HTML were
> around for 2.2.1c2 when I tried to pick it up the other night... Is there
> going to be an info release for the 2.2.1 final, or should I just leave it
> as HTML-only?
The GNU info files have been contributed, but a patch has just been
submitted that I'll try to get to in time. In the past, we've run
into problems with elisp library incompatibilies between emacs and
xemacs. We'll see how the proposed patch holds up.
Even if we aren't able to get it working, someone will probably
contribute a version. This is how all recent GNU info versions have
been prepared (like for the past four years or so).
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From goodger@users.sourceforge.net Fri Apr 5 04:33:31 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Thu, 04 Apr 2002 23:33:31 -0500
Subject: [Doc-SIG] Re: PEP 287: reStructuredText Standard Docstring Format
In-Reply-To:
References: <200204042338.XAA12847@crown.off.ekorp.com> <01c701c1dc35$5a2312f0$02010a0a@suxlap>
Message-ID:
Thank you to everyone who has commented on PEP 287, be they in favor,
well-meaning but opposed, or ill-wishing troll ;-) [1]_. I have been
otherwise busy and haven't been able to keep up with replies, but I have
read and kept a record of all the posts so far (and will, for posts yet to
come) for consideration and reference when revising the PEP. I may not be
able to offer timely replies to all the posts; ultimately the revised PEP
will serve as an umbrella reply.
.. [1] Actually, the posts opposed and even the trolls are often the most
useful, since they point out weaknesses and help strengthen the
arguments. The number of people posting *in favor* of the PEP was
roughly the same as the number opposed, but supporters tend to be
silent while protesters tend to be quite vociferous and relentless.
Over the weeks to come, I will be revising PEP 287 and related PEPs 256-258,
merging the reStructuredText_ and DPS_ projects into the renamed Docutils_
project, and producing a "road map" describing the big picture. Further
suggestions for improvement are welcome.
.. _reStructuredText: http://structuredtext.sourceforge.net/
.. _DPS: http://docstring.sourceforge.net/
.. _Docutils: http://docutils.sourceforge.net/
A special thanks must go to Richard Jones for taking up arms (or keyboard)
in my stead, and for taking the initiative to start work on `A
ReStructuredText Primer`_. In hindsight it was much needed, and it's coming
along nicely. Thanks Richard!
.. _A ReStructuredText Primer: http://mechanicalcat.net/tech/quickstart.html
--
David Goodger goodger@users.sourceforge.net Open-source projects:
- Python Docstring Processing System: http://docstring.sourceforge.net
- reStructuredText: http://structuredtext.sourceforge.net
- The Go Tools Project: http://gotools.sourceforge.net
From tony@lsl.co.uk Fri Apr 5 08:27:56 2002
From: tony@lsl.co.uk (Tony J Ibbs (Tibs))
Date: Fri, 5 Apr 2002 09:27:56 +0100
Subject: [Doc-SIG] Re: PEP 287: reStructuredText Standard Docstring Format
In-Reply-To:
Message-ID: <00cb01c1dc7b$c9a92c40$545aa8c0@lslp862.int.lsl.co.uk>
[python-dev cut from reply list]
David Goodger wrote:
> A special thanks must go to Richard Jones for taking up arms
> (or keyboard) in my stead, and for taking the initiative to
> start work on `A ReStructuredText Primer`_.
Ooh, ooh, neat stuff!
> In hindsight it was much needed, and it's coming
> along nicely. Thanks Richard!
Oh, it was obvious it was needed, but it needed someone to do it! And
the initial part of the document up now is good stuff.
I must admit I like the term "Relaxed text" - should we consider a name
change? (sowing trouble for the sake of it!)
Tibs
--
Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/
Give a pedant an inch and they'll take 25.4mm
(once they've established you're talking a post-1959 inch, of course)
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)
From fdrake@acm.org Fri Apr 5 18:30:58 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 5 Apr 2002 13:30:58 -0500
Subject: [Doc-SIG] Re: [Python-Dev] Searching Python docs
In-Reply-To: <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook>
References:
<3CAC2F2B.1950AD8F@lemburg.com>
<018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook>
<15532.48440.566653.321044@magrathea.basistech.com>
<025701c1dcb3$037af110$e000a8c0@thomasnotebook>
<3CADD457.4D821311@lemburg.com>
<03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook>
Message-ID: <15533.60770.80493.19497@grendel.zope.com>
[This is cross-posted; followups to the Doc-SIG only, please!]
Thomas Heller writes:
> Hm, in the good old days this was the only way ;-)
> because the chances where at most 50% that this stuff
> was undocumented.
I think a lot of us here remember when that 50% was less than 5%, and
there was no C API manual. ;-) I still remember seeing the checkin
message for api.tex 1.1.
> But the docs have improved in the meantime, now you
> could also use *them*.
Which brings me to the issue of a decent index. (What doesn't,
eventually?)
We currently have two documents that discuss the C API, though they
take (mostly) different approaches. The API reference has a tolerable
index (the symbols are there, at least), but the Extending & Embedding
document doesn't have enough index entries to be very useful (so
little we don't even include an index), even though it contains some
strongly overlapping information.
I *think* it might be a good idea to merge the two documents, but I'm
not certain I really like that. There is a strong need to add good
index entries to the existing Extending & Embedding document at the
very least, and a combined index would be very valuable. This is
something that's been requested for the entire set of documents on
many occaisions, and would be very handy. (Especially if we provided
a better interface for it!)
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From thomas.heller@ion-tof.com Fri Apr 5 18:53:56 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 5 Apr 2002 20:53:56 +0200
Subject: [Doc-SIG] Re: [Python-Dev] Searching Python docs
References: <3CAC2F2B.1950AD8F@lemburg.com><018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook><15532.48440.566653.321044@magrathea.basistech.com><025701c1dcb3$037af110$e000a8c0@thomasnotebook><3CADD457.4D821311@lemburg.com><03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> <15533.60770.80493.19497@grendel.zope.com>
Message-ID: <043001c1dcd3$3d1d8700$e000a8c0@thomasnotebook>
From: "Fred L. Drake, Jr."
>
> [This is cross-posted; followups to the Doc-SIG only, please!]
(Great, the next mailing list for me to subscribe :-)
>
> Which brings me to the issue of a decent index. (What doesn't,
> eventually?)
>
> We currently have two documents that discuss the C API, though they
> take (mostly) different approaches. The API reference has a tolerable
> index (the symbols are there, at least), but the Extending & Embedding
> document doesn't have enough index entries to be very useful (so
> little we don't even include an index), even though it contains some
> strongly overlapping information.
>
> I *think* it might be a good idea to merge the two documents, but I'm
> not certain I really like that. There is a strong need to add good
> index entries to the existing Extending & Embedding document at the
> very least, and a combined index would be very valuable. This is
> something that's been requested for the entire set of documents on
> many occaisions, and would be very handy. (Especially if we provided
> a better interface for it!)
IMO an index is not needed for the tutorial style manuals, and Extending
& Embedding is one of them. I'm fine with the existing indexes for the
api, lib, and ref manuals. They contain the complete entries which I have
to look up from time to time.
My experience is as follows: I used to use the HTMLHelp version of
the Python docs most of the time, mainly for context-sensitive help
from the editor: Put the cursor on the 'PyObject_GetItemString' word,
press F1, and look at the docs for this function.
This is a use case which is very common for me, and I have not yet
been able to drive a web search engine to do that for me.
The problem is that I always have to locate and download (or even build)
the HTMLHelp version separately.
Therefore I wrote my pyhelp script which does the same from the
command line, and now also as CGI script. Again, in case anyone wants
a to look at a preview:
http://starship.python.net/crew/theller/cgi-bin/pyhelp.cgi
(Any comments/suggestions/flames greatly appreciated).
Enter PyDict_ and press the search button: I find it very useful.
Note that this script (which is in an early stage) finds the index
entries from the api, lib and ref manuals.
Thomas
From guido@python.org Fri Apr 5 19:17:03 2002
From: guido@python.org (Guido van Rossum)
Date: Fri, 05 Apr 2002 14:17:03 -0500
Subject: [Doc-SIG] Re: [Python-Dev] Searching Python docs
In-Reply-To: Your message of "Fri, 05 Apr 2002 13:30:58 EST."
<15533.60770.80493.19497@grendel.zope.com>
References: <3CAC2F2B.1950AD8F@lemburg.com> <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> <3CADD457.4D821311@lemburg.com> <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook>
<15533.60770.80493.19497@grendel.zope.com>
Message-ID: <200204051917.g35JH3w15377@pcp742651pcs.reston01.va.comcast.net>
> We currently have two documents that discuss the C API, though they
> take (mostly) different approaches. The API reference has a tolerable
> index (the symbols are there, at least), but the Extending & Embedding
> document doesn't have enough index entries to be very useful (so
> little we don't even include an index), even though it contains some
> strongly overlapping information.
>
> I *think* it might be a good idea to merge the two documents, but I'm
> not certain I really like that. There is a strong need to add good
> index entries to the existing Extending & Embedding document at the
> very least, and a combined index would be very valuable. This is
> something that's been requested for the entire set of documents on
> many occaisions, and would be very handy. (Especially if we provided
> a better interface for it!)
The E&E document should be turned into a tutorial, with a bunch of
platform specific things about building. The API reference
documentation (in particular PyArg_ArgParse and Py_BuildValue) should
be moved from the E&E document to the API document.
--Guido van Rossum (home page: http://www.python.org/~guido/)
From goodger@users.sourceforge.net Tue Apr 9 01:20:09 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Mon, 08 Apr 2002 20:20:09 -0400
Subject: [Doc-SIG] Re: PEP 287: reStructuredText Standard Docstring
Format
In-Reply-To: <00cb01c1dc7b$c9a92c40$545aa8c0@lslp862.int.lsl.co.uk>
Message-ID:
Tony J Ibbs (Tibs) wrote:
> I must admit I like the term "Relaxed text" - should we consider a
> name change?
We could always add "relaxed" to the list of meanings for the "re" in
"reStructuredText", and pretend that we've hijacked the time machine.
> (sowing trouble for the sake of it!)
What else is new? :>
--
David Goodger goodger@users.sourceforge.net Open-source projects:
- Python Docstring Processing System: http://docstring.sourceforge.net
- reStructuredText: http://structuredtext.sourceforge.net
- The Go Tools Project: http://gotools.sourceforge.net
From fredrik@pythonware.com Tue Apr 9 08:12:30 2002
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 9 Apr 2002 09:12:30 +0200
Subject: [Doc-SIG] ann: xmltoys 1.0a4, featuring pythondoc
Message-ID: <00e101c1df95$eeb672a0$ced241d5@hagrid>
I just released xmltoys 1.0a4, which includes an updated
version of the javadoc-like pythondoc document parser:
http://effbot.org/downloads/index.cgi/xmltoys-1.0a4-20020407.zip
(click on download to get the code)
-- about xmltoys pythondoc
this tool uses introspection to find functions, classes, and
methods in a module, and source code scanning to extract
javadoc-style comments.
the description part is written in HTML, and converted to
XHTML fragments by the parser. additional information is
extracted from the source code, and from javadoc tags in
the comment. the result is stored in an XML infoset.
the tool comes with a relatively simple HTML converter;
a docbook converter is in the works. you can of course
also run the result through your favourite XML processor.
more information can be found here:
http://effbot.org/guides/pythondoc.htm
http://java.sun.com/j2se/javadoc/
http://java.sun.com/j2se/1.4/docs/tooldocs/solaris/javadoc.html
sample output (showing off some rather obvious bugs):
http://effbot.org/doc/
-- about the 1.0a4 release
this version has been tested with itself, PIL, and a couple of
other rather large libraries. unlike the first release, it can also
parse the python standard library without getting stuck on
None objects (but for obvious reasons, it doesn't generate
any output for the standard library...
it uses the new HTMLParser module, so the current version
requires Python 2.2. I plan to get rid of that dependency for
the next release.
enjoy /F
From fdrake@acm.org Tue Apr 9 17:30:32 2002
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 9 Apr 2002 12:30:32 -0400
Subject: [Doc-SIG] 2.2.1 docs
Message-ID: <15539.5928.855738.194157@grendel.zope.com>
The Python 2.2.1 docs have now been published on python.org, and are
listed as the "current" documentation.
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From vdesmedt@cso.ulb.ac.be Fri Apr 12 08:56:26 2002
From: vdesmedt@cso.ulb.ac.be (Vivian De Smedt)
Date: Fri, 12 Apr 2002 09:56:26 +0200
Subject: [Doc-SIG] State of xml documentation standart project.
Message-ID: <5.1.0.14.2.20020412094652.00a6adc0@cso.ulb.ac.be>
Dear all,
I have write some module documentation using LaTeX according to the
documentation standart of Python but I didn't succeed in converting them
into html pages using Latex2Html on my windows platform.
Hence I'am interested in the xml version of the documentation standart and
the existing tools that already exist to convert from:
latex to xml
xml to latex
xml to html
If they do not exist I will be glad to help writing xslt filter to produce
both html and LaTeX.
I'am sorry not to be able to find what's happening with the xml migration
project and I hope not bothering you with question I should be able to
answer alone.
Vivian.
From brian@sweetapp.com Fri Apr 12 17:56:15 2002
From: brian@sweetapp.com (Brian Quinlan)
Date: Fri, 12 Apr 2002 09:56:15 -0700
Subject: [Doc-SIG] State of xml documentation standart project.
In-Reply-To: <5.1.0.14.2.20020412094652.00a6adc0@cso.ulb.ac.be>
Message-ID: <003b01c1e242$f6371b10$445d4540@Dell2>
This is a multi-part message in MIME format.
------=_NextPart_000_003C_01C1E208.49D84310
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Vivian wrote:
> I'am sorry not to be able to find what's happening with the xml
migration
> project and I hope not bothering you with question I should be able to
> answer alone.
In order to avoid learning LaTeX for my Python extensions, I wrote a
simple XSLT stylesheet that converts my made-up XML grammar to html.
I've attached the XML, stylesheet and resulting HTML.
I too would be winning to help move us towards a standard SGML
documentation solution.
Cheers,
Brian
------=_NextPart_000_003C_01C1E208.49D84310
Content-Type: text/xml;
name="PySilverCityDocs.xml"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="PySilverCityDocs.xml"
=20
SilverCitySilverCity=20
--- Multilanguage lexical analysis package
SilverCity is a library that can provide lexical analysis for over 20 =
different=20
programming languages. SilverCity is packaged as both a C++ library and =
as a=20
Python extension. This documentation applies to the Python =
extension.
At this point I'd like to acknoledge that this documentation is =
incomplete. Writting
isn't a hobby of mine. So if you need any help, just let me know at =
<brian@sweetapp.com>.
ScintillaScintilla is the open-source source editing =
component
upon which SilverCity is builtPython tokenize modulePython's built-in lexical scanner for Python source =
code.
=0D Module Contentsfind_lexer_module_by_idid
The find_lexer_module_by_id function
returns a LexerModule object given an integer
constant. These constants are defined in the=20
ScintillaConstants module.
A ValueError is raised if the
provided constant does not map to a =
LexerModule.
find_lexer_module_by_namename
The find_lexer_module_by_name function
returns a LexerModule object given it's name.
A ValueError is raised if no =
LexerModule
has the given name
WordListkeywords
Create a new WordList instance.=20
This class is used by the=20
LexerModule class to determine which
words should be lexed as keywords.
keywords should be
a string containing keywords separated by spaces=20
e.g. "and assert break class..."
WordList objects have no methods. They simply =
act as placeholders for=20
language keywords.
PropertySetproperties
Create a new PropertySet instance.=20
This class is used by the=20
LexerModule class to determine various
lexer options. For example, the 'styling.within.preprocessor'
property determines if the C lexer should use a single or
multiple lexical states when parsing C preprocessor expressions.
properties should be
a dictionary of lexer options.
LexerModule objects
The LexerModule class provides a single method:
get_number_of_wordlists
Returns the number of WordLists that the lexer =
requires.=20
This is the number of WordLists that must be
passed to the tokenize_by_style.
If the number of required WordLists cannot be determined, a =
ValueError,
is raised
tokenize_by_stylesourcekeywordspropertysetfunc
Lexes the provided source code into a list of tokens. Each token =
is a dictionary with the following
keys:
Key
Value
style
The lexical style of the token =
e.g. 11
text
The text of the token e.g. =
'import'
start_index
The index in =
source where the token begins e.g. 0
end_index
The index in =
source where the token ends e.g. 5
start_column
The column position =
(0-based) where the token begins e.g. 0
end_column
The column position =
(0-based) where the token ends e.g. 5
start_line
The line position =
(0-based) where the token begins e.g. 0
end_line
The line position (0-based) =
where the token ends e.g. 0
=20
=20
source is a string containing the source code.
keywords is a list of WordList =
instances.
The number of WordLists that should be passed =
depends on the particular=20
LexerModule.
=20
propertyset is a PropertySet instance.
The relevant properties are dependant on the particular =
LexerModule.
If the optional func argument is used, it must be =
a callable object. It will
be called, using keyword arguments, for each token found in the =
source. Since additional
keys may be added in the future, it is recommended that =
additional keys be collected e.g.
import SilverCity
from SilverCity import ScintillaConstants
=20
def func(style, text, start_column, start_line, =
**other_args):=20
if style =3D=3D ScintillaConstants.SCE_P_WORD and text =
=3D=3D 'import':
print 'Found an import statement at (%d, %d)' % =
(start_line + 1, start_column + 1)
=20
=20
keywords =3D =
SilverCity.WordList(SilverCity.Keywords.python_keywords)
properties =3D SilverCity.PropertySet()
lexer =3D =
SilverCity.find_lexer_module_by_id(ScintillaConstants.SCLEX_PYTHON)
=20
lexer.tokenize_by_style(source_code, keywords, properties, =
func)
WordList objects
WordList objects have no methods. They simply act as =
placeholders for=20
language keywords.
PropertySet objects
PropertySet objects have no methods. They act as =
dictionaries were the
names of the properties are the keys. All keys must be strings, values =
will be converted to strings
upon assignment i.e. retrieved values will always be strings. There is =
no mechanism to delete
assigned keys.
Different properties apply to different languages. The following =
table is a complete
list of properties, the language that they apply to, and their =
meanings:
Property
Language
Values
asp.default.language
HTML
Sets the default language for ASP scripts:
0 =3D> None
1 =3D> JavaScript
2 =3D> VBScript
3 =3D> Python
4 =3D> PHP
5 =3D> XML-based
styling.within.preprocessor
C++
Determines if all preprocessor instruments should be lexed =
identically or
if subexpressions should be given different lexical states:
0 =3D> Same
1 =3D> Different
tab.timmy.whinge.level
Python
The property value is a bitfield that causes different types of =
incorrect=20
whitespace characters to cause there lexical states to be incremeted by =
64:
0 =3D> no checking
1 =3D> check for correct indenting
2 =3D> check for literal tabs
4 =3D> check for literal spaces used as tabs
8 =3D> check for mixed tabs and spaces
Example PropertySet usage:
import SilverCity
=20
propset =3D SilverCity.PropertySet({'styling.within.preprocessor' : =
0})
propset['styling.within.preprocessor'] =3D 1 # changed my mind
Stuff that should be documented better
The ScintillaConstants module contains a list of =
lexer identifiers
(used by find_lexer_module_by_id) and lexical =
states for each
LexerModule. You should take a look at this module to =
find the states
that are useful for your programming language.
The Keywords module contains lists of keywords that =
can be
used to create WordList objects.
There are also some modules that package =
tokenize_by_style
into a class that offers a visitor pattern (think SAX). You don't have =
to worry about these
modules if you don't want to. But, if you do, they are all written in =
Python so you can probably
muddle through.
Note that some lexer that are supported by Scintilla, are not =
supported by=20
SilverCity. This is because I am lazy. Any =
contributions are welcome
(and should be pretty easy to make).
SilverCity is a library that can provide lexical analysis for over 20 =
different programming languages. SilverCity is packaged as both a C++ =
library=20
and as a Python extension. This documentation applies to the Python=20
extension.
At this point I'd like to acknoledge that this documentation is =
incomplete.=20
Writting isn't a hobby of mine. So if you need any help, just let me =
know at=20
<brian@sweetapp.com>.
The find_lexer_module_by_id function returns a=20
LexerModule object given an integer constant. These constants =
are=20
defined in the ScintillaConstants module.=20
A ValueError is raised if the provided constant does not =
map to a=20
LexerModule.
Create a new PropertySet instance. This class is used by =
the=20
LexerModule class to determine various lexer options. For =
example,=20
the 'styling.within.preprocessor' property determines if the C lexer =
should=20
use a single or multiple lexical states when parsing C preprocessor=20
expressions.=20
properties should be a dictionary of lexer options.=20
Lexes the provided source code into a list of tokens. Each token =
is a=20
dictionary with the following keys:=20
Key
Value
style
The lexical style of the token =
e.g.=20
11
text
The text of the token e.g. =
'import'
start_index
The index in source =
where the=20
token begins e.g. 0
end_index
The index in source =
where the=20
token ends e.g. 5
start_column
The column position (0-based) =
where the=20
token begins e.g. 0
end_column
The column position (0-based) =
where the=20
token ends e.g. 5
start_line
The line position (0-based) =
where the=20
token begins e.g. 0
end_line
The line position (0-based) =
where the=20
token ends e.g. 0
source is a string containing the source code.=20
keywords is a list of WordList instances. The =
number of=20
WordLists that should be passed depends on the particular=20
LexerModule. propertyset is a PropertySet =
instance. The relevant properties are dependant on the particular=20
LexerModule.
If the optional func argument is used, it must be a =
callable=20
object. It will be called, using keyword arguments, for each token =
found in=20
the source. Since additional keys may be added in the future, it is=20
recommended that additional keys be collected e.g.=20
import SilverCity
from SilverCity import ScintillaConstants
=20
def func(style, text, start_column, start_line, =
**other_args):=20
if style =3D=3D ScintillaConstants.SCE_P_WORD and text =
=3D=3D 'import':
print 'Found an import statement at (%d, %d)' % =
(start_line + 1, start_column + 1)
=20
=20
keywords =3D =
SilverCity.WordList(SilverCity.Keywords.python_keywords)
properties =3D SilverCity.PropertySet()
lexer =3D =
SilverCity.find_lexer_module_by_id(ScintillaConstants.SCLEX_PYTHON)
=20
lexer.tokenize_by_style(source_code, keywords, properties, =
func)
PropertySet objects have no methods. They act as =
dictionaries were=20
the names of the properties are the keys. All keys must be strings, =
values will=20
be converted to strings upon assignment i.e. retrieved values will =
always be=20
strings. There is no mechanism to delete assigned keys.
Different properties apply to different languages. The following =
table is a=20
complete list of properties, the language that they apply to, and their=20
meanings:
Property
Language
Values
asp.default.language
HTML
Sets the default language for ASP =
scripts: 0 =3D> None 1 =3D> JavaScript 2 =3D> =
VBScript 3=20
=3D> Python 4 =3D> PHP 5 =3D> =
XML-based
styling.within.preprocessor
C++
Determines if all preprocessor =
instruments=20
should be lexed identically or if subexpressions should be given =
different=20
lexical states: 0 =3D> Same 1 =3D> =
Different
tab.timmy.whinge.level
Python
The property value is a bitfield =
that=20
causes different types of incorrect whitespace characters to cause =
there=20
lexical states to be incremeted by 64: 0 =3D> no =
checking 1 =3D>=20
check for correct indenting 2 =3D> check for literal =
tabs 4 =3D>=20
check for literal spaces used as tabs 8 =3D> check for mixed =
tabs and=20
spaces
Example PropertySet usage:=20
import SilverCity
=20
propset =3D SilverCity.PropertySet({'styling.within.preprocessor' : =
0})
propset['styling.within.preprocessor'] =3D 1 # changed my mind
The ScintillaConstants module contains a list of lexer =
identifiers=20
(used by find_lexer_module_by_id) and lexical states for each=20
LexerModule. You should take a look at this module to find the =
states=20
that are useful for your programming language.
The Keywords module contains lists of keywords that can be =
used to=20
create WordList objects.
There are also some modules that package tokenize_by_style =
into a=20
class that offers a visitor pattern (think SAX). You don't have to worry =
about=20
these modules if you don't want to. But, if you do, they are all written =
in=20
Python so you can probably muddle through.
Note that some lexer that are supported by Scintilla, are not =
supported by=20
SilverCity. This is because I am lazy. Any contributions are =
welcome=20
(and should be pretty easy to make).
------=_NextPart_000_003C_01C1E208.49D84310--
From goodger@users.sourceforge.net Fri Apr 19 05:01:50 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Fri, 19 Apr 2002 00:01:50 -0400
Subject: [Doc-SIG] DPS & reStructuredText 0.4 Released
Message-ID:
Release 0.4 of each of the Docstring Processing System and reStructuredText
markup parser projects have been posted to their project sites (URLs in my
.sig below). These releases are the final ones for these projects. The two
projects will be merged and renamed to "Docutils"; a SourceForge project is
being constructed at http://docutils.sourceforge.net/, and a 0.1 release of
Docutils will follow shortly.
Once the DPS and reStructuredText projects are merged and renamed, they will
become inactive. So this is really a contractual obligation
non-announcement; you might as well wait a day or three for Docutils 0.1.
Quick links to the downloads:
- http://prdownloads.sf.net/structuredtext/restructuredtext-0.4.tar.gz
- http://prdownloads.sf.net/docstring/dps-0.4.tar.gz
There has been a lot of progress since the last release, long ago. The
parser is construct-complete, there's a standalone input file reader, and a
simple HTML/CSS writer. There are a few simple front-ends in
restructuredtext/tools; see its README.txt.
There is still much work to be done. If you would like to be added as a
developer to the new Docutils project, please let me know (I've already
added some people as developers).
--
David Goodger goodger@users.sourceforge.net Open-source projects:
- Python Docstring Processing System: http://docstring.sourceforge.net
- reStructuredText: http://structuredtext.sourceforge.net
- The Go Tools Project: http://gotools.sourceforge.net
From goodger@users.sourceforge.net Sat Apr 20 20:10:24 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Sat, 20 Apr 2002 15:10:24 -0400
Subject: [Doc-SIG] ANN: Docutils 0.1 Released
Message-ID:
The purpose of the Docutils project is to create a set of tools for
processing plaintext documentation into useful formats, such as HTML,
XML, and TeX. It is the subject of PEPs 256 & 258. Docutils
currently supports input from standalone files; soon it will process
PEPs, and eventually it will support inline documentation from Python
modules and packages. Docutils uses the reStructuredText markup, the
subject of PEP 287; other markups are possible. It produces simple
HTML for CSS; other formats will become available in time.
Quick link to the download:
http://prdownloads.sf.net/docutils/docutils-0.1.tar.gz
Docutils home page: http://docutils.sourceforge.net/
To subscribe to the mailing lists:
- Development discussions (docutils-develop@lists.sourceforge.net):
http://lists.sourceforge.net/lists/listinfo/docutils-develop
- CVS checkin messages:
http://lists.sourceforge.net/lists/listinfo/docutils-checkins
High-level discussions take place on the Python Doc-SIG mailing list:
http://mail.python.org/mailman/listinfo/doc-sig,
mailto:Doc-SIG@python.org
A ReStructuredText Primer (gentle introduction):
http://docutils.sf.net/docs/rst/quickstart.html
Quick reStructuredText (user reference):
http://docutils.sf.net/docs/rst/quickref.html
There has been a lot of progress lately. The reStructuredText parser
is feature-complete, there's a standalone input file reader, and a
simple HTML/CSS writer. The web site's HTML files are generated from
reStructuredText source (the quickref being the only exception).
There are a few simple front-ends in docutils/tools; see the README:
http://docutils.sf.net/README.html.
There is still much work to be done. Contributors are welcome!
Release 0.1 (2002-04-20)
========================
This is the first release of Docutils, merged from the now inactive
reStructuredText_ and `Docstring Processing System`_ projects. For
the pre-Docutils history, see the `reStructuredText HISTORY`_ and
`DPS HISTORY`_ files.
.. _reStructuredText: http://structuredtext.sf.net/
.. _Docstring Processing System: http://docstring.sf.net/
.. _reStructuredText HISTORY:
http://structuredtext.sf.net/HISTORY.html
.. _DPS HISTORY: http://docstring.sf.net/HISTORY.html
General changes: renamed 'dps' package to 'docutils'; renamed
'restructuredtext' subpackage to 'rst'; merged the codebases; merged
the test suites (reStructuredText's test/test_states renamed to
test/test_rst); and many modifications required to make it all work.
--
David Goodger Open-source projects:
- Python Docutils: http://docutils.sourceforge.net/
- The Go Tools Project: http://gotools.sourceforge.net/
From goodger@users.sourceforge.net Mon Apr 29 02:40:33 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Sun, 28 Apr 2002 21:40:33 -0400
Subject: [Doc-SIG] Auto-Numbered Enumerated Lists (reStructuredText)
Message-ID:
[The following is an excerpt from
http://docutils.sf.net/spec/rst/alternatives.html (working version, not
checked in yet). It was prompted by a suggestion by Fred Bremmer, which
became alternative 3. I'm looking for comments, pro or con, on all
aspects. Should this be implemented? (The implementation itself is
trivial.) Which syntax is best? Which behavior? Any other suggestions?
TIA.]
The advantage of auto-numbered enumerated lists would be similar to
that of auto-numbered footnotes: lists could be written and rearranged
without having to manually renumber them. The disadvantages are also
the same: input and output wouldn't match exactly; the markup may be
ugly or confusing (depending on which alternative is chosen).
1. Use the "#" symbol. Example::
#. Item 1.
#. Item 2.
#. Item 3.
Advantage: simple, explicit. Disadvantage: enumeration sequence
cannot be specified (limited to arabic numerals); ugly.
2. As a variation on #1, first initialize the enumeration sequence?
For example::
a) Item a.
#) Item b.
#) Item c.
Advantages: simple, explicit, any enumeration sequence possible.
Disadvantages: ugly; perhaps confusing with mixed concrete/abstract
enumerators.
3. Alternative suggested by Fred Bremmer, from experience with MoinMoin::
1. Item 1.
1. Item 2.
1. Item 3.
Advantages: enumeration sequence is explicit (could be multiple
"a." or "(I)" tokens). Disadvantages: perhaps confusing; otherwise
erroneous input (e.g., a duplicate item "1.") would pass silently,
either causing a problem later in the list (if no blank lines
between items) or creating two lists (with blanks).
Take this input for example::
1. Item 1.
1. Unintentional duplicate of item 1.
2. Item 2.
Currently the parser will produce two list, "1" and "1,2" (no
warnings, because of the presence of blank lines). Using Fred's
notation, the current behavior is "1,1,2 -> 1 1,2" (without blank
lines between items, it would be "1,1,2 -> 1 [WARNING] 1,2"). What
should the behavior be with auto-numbering?
Fred has produced a patch__, whose initial behavior is as follows::
1,1,1 -> 1,2,3
1,2,2 -> 1,2,3
3,3,3 -> 3,4,5
1,2,2,3 -> 1,2,3 [WARNING] 3
1,1,2 -> 1,2 [WARNING] 2
(After the "[WARNING]", the "3" would begin a new list.)
I have mixed feelings about adding this functionality to the spec &
parser. It would certainly be useful to some users (myself
included; I often have to renumber lists). Perhaps it's too
clever, asking the parser to guess too much. What if you *do* want
three one-item lists in a row, each beginning with "1."? You'd
have to use empty comments to force breaks. Also, I question
whether "1,2,2 -> 1,2,3" is optimal behavior.
In response, Fred came up with "a stricter and more explicit rule
[which] would be to only auto-number silently if *all* the
enumerators of a list were identical". In that case:
1,1,1 -> 1,2,3
1,2,2 -> 1,2 [WARNING] 2
3,3,3 -> 3,4,5
1,2,2,3 -> 1,2 [WARNING] 2,3
1,1,2 -> 1,2 [WARNING] 2
Should any start-value be allowed ("3,3,3"), or should
auto-numbered lists be limited to begin with ordinal-1 ("1", "A",
"a", "I", or "i")?
__ http://sourceforge.net/tracker/index.php?func=detail&aid=548802
&group_id=38414&atid=422032
--
David Goodger Open-source projects:
- Python Docutils: http://docutils.sourceforge.net/
- The Go Tools Project: http://gotools.sourceforge.net/
From tony@lsl.co.uk Mon Apr 29 09:55:48 2002
From: tony@lsl.co.uk (Tony J Ibbs (Tibs))
Date: Mon, 29 Apr 2002 09:55:48 +0100
Subject: [Doc-SIG] Auto-Numbered Enumerated Lists (reStructuredText)
In-Reply-To:
Message-ID: <012201c1ef5b$a7fbe450$545aa8c0@lslp862.int.lsl.co.uk>
Another alternative for auto-numbered lists, sort-of similar to the way
we handle footnotes at the moment, is to allow::
#1. First item.
#3. Aha - I edited this in later.
#2. Second item.
This requires the user to ensure that list items are still unique -
probably a good thing. I would suggest that it *not* be possible to have
two adjacent lists with this mechanism - so the problems David describes
with the list::
1. Item 1.
1. Unintentional duplicate of item 1.
2. Item 2.
would not be allowed - one would have to place a comment (or some other
"breaking" text) between the two item 1.s to get two lists. I don't see
that as a particularly onerous restriction.
Presumably one would also allow::
#a. The first item
#1. This *is* a new list.
to mean two lists, though.
As a side-effect, would we then allow people to assume that such items
could be referred to? Consider the following::
We have a list.
1. The first item
2. The second item
3. The third item
In item 2, we see the middle item.
With autonumbering, such a textual reference is more problematical - but
maybe we could allow::
We have a list.
#1. The first item
#3. The second item
#2. The third item
In item #3_, we see the middle item.
(and no, I realise that syntax is likely to be wrong, but it's the
*concept* I'm interested in).
On the other hand, I'm actually agnostic on whether we need autonumbered
lists at all - but then I'm the sort of person who will go back and
adjust indentation in a long document to make it look pretty, anyway, so
I'm maybe not the best person to judge!
Tibs
--
Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/
Well we're safe now....thank God we're in a bowling alley.
- Big Bob (J.T. Walsh) in "Pleasantville"
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)
From pinard@iro.umontreal.ca Mon Apr 29 18:07:43 2002
From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard)
Date: 29 Apr 2002 13:07:43 -0400
Subject: [Doc-SIG] Re: [development doc updates]
In-Reply-To: <15532.63699.122156.945015@grendel.zope.com>
References: <20020404225927.530B318EAD1@grendel.zope.com>
<20020404174752.M16962@tummy.com>
<15532.63699.122156.945015@grendel.zope.com>
Message-ID:
[Fred L. Drake, Jr.]
> The GNU info files have been contributed, but a patch has just been
> submitted that I'll try to get to in time. In the past, we've run
> into problems with elisp library incompatibilies between emacs and
> xemacs. We'll see how the proposed patch holds up.
If the Emacs Lisp incompatibilities come from the Emacs/XEmacs Texinfo
formatting modules, I would guess you might be better off using the
`makeinfo' tool instead, which is much more speedy, anyway. `makeinfo' (as
well as the stand-alone `info' reader) are part of the Texinfo distribution
release. I did not closely follow recent evolution in that area, however,
so please take my comment with a grain of salt.
--
Fran�ois Pinard http://www.iro.umontreal.ca/~pinard
From goodger@users.sourceforge.net Tue Apr 30 04:31:48 2002
From: goodger@users.sourceforge.net (David Goodger)
Date: Mon, 29 Apr 2002 23:31:48 -0400
Subject: [Doc-SIG] Auto-Numbered Enumerated Lists (reStructuredText)
In-Reply-To: <012201c1ef5b$a7fbe450$545aa8c0@lslp862.int.lsl.co.uk>
Message-ID:
Tony J Ibbs (Tibs) wrote:
> Another alternative for auto-numbered lists, sort-of similar to the
> way we handle footnotes at the moment, is to allow::
>
> #1. First item.
> #3. Aha - I edited this in later.
> #2. Second item.
I've added this as syntax alternative 4.
> This requires the user to ensure that list items are still unique -
> probably a good thing
That doesn't seem like a good thing to me. We'd get the ugly syntax
of the first two alternatives plus the inconvenience of having to
account for item numbers, without much convenience gain (just random
ordering).
It might fly if we modify this proposal like so:
- Simply prepend a "#" to a standard list enumerator to indicate
auto-enumeration.
- The numbers (or letters) of the enumerators themselves are not
significant, except:
- as a sequence indicator (arabic, roman, alphabetic),
- and perhaps as start value (first list item)
> I would suggest that it *not* be possible to have two adjacent lists
> with this mechanism
It'll happen though. A system message (warning or error) would be
generated; I assume that's what you mean.
> so the problems David describes with the list::
>
> 1. Item 1.
>
> 1. Unintentional duplicate of item 1.
>
> 2. Item 2.
>
> would not be allowed
Actually, there are no "problems" with this in the current
spec/parser. It results in two lists, "1" and "1,2". Were the blank
lines to be omitted between items, a warning would be generated
between the two lists. Under Fred's proposed semantics, the list
would be parsed as "1,2" and "2".
> As a side-effect, would we then allow people to assume that such
> items could be referred to?
That would be tricky, and going too far I think. Either all
auto-numbered lists in a document would need to have unique labels, or
some sort of scoping rules would be required: would a list item
reference refer to the preceding list or the following list? I'm
loathe to introduce such rules. A preferable solution would be to use
the existing hyperlink constructs to refer to the body of a list item;
the item number couldn't be transferred, but a live link could be
created.
Auto-enumerated lists are only a marginal convenience feature; the
syntax and semantics should be simple and clean. Too complex and it's
no longer convenient enough to justify its existence.
--
David Goodger Open-source projects:
- Python Docutils: http://docutils.sourceforge.net/
- The Go Tools Project: http://gotools.sourceforge.net/