The impact of '@' on Leo
The mandate for this post is the following from GvR:
Please bring up the Leo issues in Python-dev.
The major concern I have with '@' syntax as it relates to Leo is the potential for ongoing and never-ending conflicts between Leo's directives and @x constructs in Python. Leo presently supports the following directives: @ (followed by one or more whitespace characters) @all, @asis, @c, @code, @color, @comment, @delims, @doc, @encoding, @end_raw, @file, @first, @header, @ignore, @killcolor, @language, @last, @lineending, @nocolor, @noheader, @noref, @nosent, @nowrap, @others, @pagewidth, @path, @quiet, @raw, @root, @root-code, @root-doc, @silent, @tabwidth, @terse, @thin, @unit, @verbose, @wrap Worse, Leo allows plugins to define their own directives. There is already a plugin that defines @wiki. Furthermore, @run, @test and @suite are also used in special contexts. Leo's users could probably live with a small, fixed set of @x constructs in Python. But if the Python programmer can create new @x identifiers then they have a big, big problem. What happens when another Python @x construct conflicts with one of these directives or some other directive? These directives act as a markup meta-language. Many of these directives are found interspersed in the code: a conflict with a construct in Python would have most unhappy consequences. To make matters worse, one of the _best_ choices I ever made in Leo was to remove all "escape" conventions regarding '@' (and other noweb constructs). This took a bit of courage, but it has turned out to be clearly the right choice. But if Python can now generate arbitrary @x strings Leo will have to do something really drastic. There are a number of markup languages that use '@' to from the underlying language (Python) to the markup language. I was always under the impression that the clear statement in the Reference Manual that at signs are invalid everywhere (except in comments and strings) was a clear signal of an intention to keep those symbols for other valid purposes. I do hope this won't change. Edward P.S. I recently wrote a pretty-printer for Python using the tokenize module. What will be the status of the '@' token? Is it still an "errortoken" ? Sure, this is a minor point, but it _will_ affect other tools. EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html --------------------------------------------------------------------
Edward K. Ream wrote:
The major concern I have with '@' syntax as it relates to Leo is the potential for ongoing and never-ending conflicts between Leo's directives and @x constructs in Python.
Can you give an example of a file that would be difficult to understand under that change? If you allow @foo inside a Python file, it is not Python anymore, is it? Regards, Martin
The major concern I have with '@' syntax as it relates to Leo is the potential for ongoing and never-ending conflicts between Leo's directives and @x constructs in Python.
Can you give an example of a file that would be difficult to understand under that change? If you allow @foo inside a Python file, it is not Python anymore, is it?
There are two files involved. The .leo file (an xml file) contains the "marked up" code. Leo turns this file into a visual outline containing zero or more "derived files" that could be written in any language. Here is the entire top-level node in the outline that represents the derived file leoApp.py: [Node starts] @first # -*- coding: utf-8 -*- @language python @tabwidth -4 import leoGlobals as g import os import sys class LeoApp: """A class representing the Leo application itself. Ivars of this class are Leo's global variables.""" @others [Node ends] - The [Node starts] and [Node ends] are _not_ part of the node, they delimit the node in this email. - The @first directive makes sure that the first line of leoApp.py is # -*- coding: utf-8 -*- - The @language python and @tabwidth directives tell Leo how to syntax color the text and how to handle tabs. - The @others directive tells Leo to insert all descendent nodes. In this case, each descendent node contains a single method of the LeoApp class. - There are other details to Leo's markup, but this gives you the general feel. Everything else is plain Python. When saving the .leo file, Leo produces leoApp.py following the rules of the markup. In particular, Leo writes all descendent nodes in place of the @others directive, indented the same as the @others directive. Most of these descendent nodes will contain few or no Leo directives: descendent nodes inherit most directives from their ancestors. The problem is that Leo doesn't really understand _any_ language, except to colorize it. Most Leo directives must start with @ in the leftmost column, but the @others and @all directives may contain leading whitespace. If @foo is both a valid Python construct and a valid Leo directive Leo will treat @foo as a Leo directive even if it is "supposed" to be Python. If @foo is not a valid Leo directive, Leo creates a "verbatim" sentinel line in the derived file to indicate that the next line isn't a directive, even though it sorta looks like one. Like this: #@verbatim @foo This is basically how Leo handle's languages with @ signs. There is no problem until @x in the language conflicts with a Leo directive, in which case Leo doesn't have any way of distinguishing which usage was intended. As I said in another post, Leo doesn't have escape conventions involving @ signs, and that has turned out well. There have been requests to allow strings other than "@" to delimit Leo directives, and that will probably happen, but in general Leo really would rather stick to "@" Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html --------------------------------------------------------------------
In article <005e01c47b2d$9c0dee80$6700a8c0@computer>, "Edward K. Ream" <edreamleo@charter.net> wrote:
The major concern I have with '@' syntax as it relates to Leo is the potential for ongoing and never-ending conflicts between Leo's directives and @x constructs in Python.
I don't know what Leo is, but the usual solution to this sort of problem is to introduce some kind of quoting convention, e.g. @@ gets passed through as a single @ to Python, any other @something gets treated as a Leo directive. Is there some reason that wouldn't work? -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/
I don't know what Leo is, but the usual solution to this sort of problem is to introduce some kind of quoting convention, e.g. @@ gets passed through as a single @ to Python, any other @something gets treated as a Leo directive. Is there some reason that wouldn't work?
Yes, in Leo's case there is a strong reason for not using an "escape" or "quoting" mechanism. As I said in my original post, not using an escape mechanism turns out to be the correct choice. The basic problem is that there are two (or is it three?) views of the data in the Leo world: the view presented to the user in the outline, the data in the .leo file, and the data in the derived files. Leo has a mechanism for automatically converting data in derived files into the one or two other forms. This is call "untangling". It turns out that it is extremely difficult to handle escapes properly when untangling. I know this sounds weird, and it's true nevertheless. If the @ syntax goes through, Leo will have to allow users to define their own lead-in string, and the format of derived files will have to change a bit to specify what the lead-in string is. It's not a huge deal, I think. It would surely be much easier than changing how Leo tangles or untangles code. BTW, I find the arguments for list-after-def to be persuasive, viz., that annotations should clearly be part of a function. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html --------------------------------------------------------------------
"Edward K. Ream" <edreamleo@charter.net>:
I was always under the impression that the clear statement in the Reference Manual that at signs are invalid everywhere (except in comments and strings) was a clear signal of an intention to keep those symbols for other valid purposes.
I've only ever regarded that as a statement about the status quo, not any kind of promise about the future. But maybe it would be a good idea to explicitly reserve some character (perhaps only at the beginning of a line). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Aug 5, 2004, at 10:41 PM, Greg Ewing wrote:
"Edward K. Ream" <edreamleo@charter.net>:
I was always under the impression that the clear statement in the Reference Manual that at signs are invalid everywhere (except in comments and strings) was a clear signal of an intention to keep those symbols for other valid purposes.
I've only ever regarded that as a statement about the status quo, not any kind of promise about the future.
But maybe it would be a good idea to explicitly reserve some character (perhaps only at the beginning of a line).
You mean like # ? :) -bob
On Thursday 05 August 2004 10:41 pm, Greg Ewing wrote:
But maybe it would be a good idea to explicitly reserve some character (perhaps only at the beginning of a line).
Bob Ippolito responded:
You mean like # ?
Yeah, like that. ;-) That's always accepted as the first non-blank character on a non-continuation line, and the only thing Python ever cares about after that are the encoding turd and a newline. Anything after the newline is, well, another line. I've really only managed to look at Leo much once, and Edward was sitting next to me at the time. Leo doesn't store Python code in a Leo document as any sort of embedded object that's labelled as Python code as far as I can tell (though it may in it's data model); it just more text, and can have @directives like any other text. I think this is inherently a risky approach to the data; there needs to be at least a way to say that some portion of the data is some other text format embedded into the Leo document. I'm sure Edward can find a way to make things work for Leo users, but I hope the .leo files have some sort of format version indicator to make upgrading relatively easy for Leo users. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
I've really only managed to look at Leo much once, and Edward was sitting next to me at the time.
Leo doesn't store Python code in a Leo document as any sort of embedded object that's labeled as Python code as far as I can tell (though it may in it's data model); it just more text, and can have @directives like any other text. I think this is inherently a risky approach to the data; there needs to be at least a way to say that some portion of
Hi Fred. Sitting next to you at the sprint was one of the most enlightening things I have done in a long time. I hope the next time we sprint together you'll be able to use Leo much more easily with cvs. I've spent that last six months on @thin, which should help a lot... the
data is some other text format embedded into the Leo document.
.leo files are XML files. The only thing that matters in that context is that the xml escapes are handled properly. What matters for the integrity of data in derived files is that uses don't mess with the so-called sentinel lines. These lines "carry" the outline structure. Destroy those lines, destroy something essential. Leo has a number of ways of recovering partially from such destruction, but the fact remains that uses insert, delete, fold or mutilate sentinel lines at their own risk. It's as simple as that. Sentinel lines start with #@ at present, and it's conceivable that they will always do so, even if @ gets used for something in Python.
I'm sure Edward can find a way to make things work for Leo users, but I hope the .leo files have some sort of format version indicator to make upgrading relatively easy for Leo users.
Actually, both .leo files and derived files have various format indicators. However, the format of .leo files hasn't changed significantly enough to need a new number in years. Allowing different lead-in strings for Leo directives will have absolutely no effect on the format of .leo files. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo@charter.net Leo: Literate Editor with Outlines Leo: http://webpages.charter.net/edreamleo/front.html --------------------------------------------------------------------
participants (6)
-
"Martin v. Löwis" -
Bob Ippolito -
David Eppstein -
Edward K. Ream -
Fred L. Drake, Jr. -
Greg Ewing