On 3/19/06, Christopher Armstrong <radeex@gmail.com> wrote:

On 3/18/06, James Schend <blakeyrat@gmail.com> wrote:

When I run that example, I get the message that

...

twisted.protocols.telnet is deprecated and I should instead be using twisted.conch.telnet.

Is there something I'm missing here, or is Twisted telling me that I

...

should be using an unstable <1.0 library with no documentation that's not even part of the core install instead of a 2.2 version library with good documentation that *is* part of the core install? Is this deprecated warning in the wrong spot? I don't get it.

The telnet protocol implementation was simply moved to Conch. That particular code is no more or less stable and documented than it was in its old location. Also, if you're downloading tarballs, you should be getting the full Twisted tarball, not the individual core/subproject tarballs. Those individual ones are for embedders and OS packagers, basically. (The old web site didn't make this as clear, and the new one encourages you to download the full Twisted package).

I'm sorry, it appears I'm mistaken. I was thinking that the telnet protocol implementation moved to twisted.conch in the Great Split of Two Point Oh, but I see that this was something else. The implementation in conch is definitely different (and probably better), but I don't think my comment about the state of documentation and stability is any less true. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/

On Sun, 19 Mar 2006 10:08:47 -0800, James Schend <blakeyrat@gmail.com> wrote:

I'm still not convinced. If you look at the API reference under Conch:

http://twistedmatrix.com/documents/current/api/twisted.conch.telnet.html

And the API refrerence under Protocols:

http://twistedmatrix.com/documents/current/api/twisted.protocols.telnet.Teln...

There's a HUGE difference in the amount of documentation available.

Yes. There seems to be much more documentation for twisted.conch.telnet.

Just learning this stuff for the first time, I need as much documentation as I can get... all the examples, and all the API docs, seem to be in twisted.protocols.telnet. And of course I'd much rather use something that's at version 2.2 than something that's at version 0.6.

Don't attach too much significance to these numbers. It would be great if versioning actually conveyed some information, but it hardly does. The only thing you can reliably take away from a version number is whether or not it is larger or smaller than another version /for the same piece of software/. This loosely correlates with whether it is newer or older than another release of that software, little else. In particular, you cannot compare version numbers of different pieces of software meaningfully.

If conch is still unstable and hasn't hit 1.0, why is the old interface deprecated?

Because the old interface is quirky and incomplete and the implementation is unmaintained (and basically unfixable without breaking backwards compatibility).

It seems silly to deprecate the old interface if the new one, being unstable, might change and require being deprecated again. I'm sorry, I just don't understand.

The deprecation is an indication of our future plans. If you are more worried about the twisted.conch.telnet API changing than you are about the incompleteness and unmaintainedness of twisted.protocols.telnet, by all means, use twisted.protocols.telnet. It's not a "warning" for nothing.

(FYI, I did download the entire tarball, but I only installed the "core" because I figured that that was all I needed. I didn't anticipate that "correct" telnet interface had moved because, again, this isn't documented in the API reference, which I looked at before installing.)

You say that Conch is better... how is it better? Is twisted.protocol.telnet going away in the near future?

As soon as reasonably possible.

What the heck should I be using?

twisted.conch.telnet.

Other than the deprecated warning, I'm inclined to use the "old" twisted.protocols.telnet interface which at least has examples and documented functions.

I'm confused by your repeated claims of the absense of documentation for twisted.conch.telnet. <http://twistedmatrix.com/documents/current/api/twisted.conch.telnet.html> lists two interfaces, six exception classes, and five protocol classes. The API documentation pages for the two interfaces alone document the eleven public methods of twisted.conch.telnet completely, and <http://twistedmatrix.com/projects/conch/documentation/examples/> gives numerous examples of both custom telnet and ssh servers.

Thanks again for your help. It looks like I found your toolset in the midst of some huge confusing rewrite or something. :) Just my luck.

Things are pretty much always like this. The only possible hope for perfect documentation is to have a dead project. I much prefer an active one. Jean-Paul

On Sun, 19 Mar 2006 10:54:27 -0800, James Schend <blakeyrat@gmail.com> wrote:

http://twistedmatrix.com/documents/current/api/twisted.conch.telnet.html

It lists the classes, and then right next to (almost all of) them it says "undocumented."

How can the two interfaces document the public method completely if they're undocumented?

Did you try clicking on any links on that page?

Thanks for your second link, however, I hadn't found that page during my surfing of your site. It looks useful.

It should probably be displayed somewhere more prominently on the site. For some reason, someone has added a link to the conch howtos to the front page, which may leave readers with the incorrect impression that is all of the conch documentation.

I still don't understand why you'd deprecate a stable interface in preference to an *unstable* one.

Because I have no interest in maintaining the stable interface since it is so narrowly useful.

What happens if conch changes in some way and needs to be deprecated again? It seems to me like it's a "damned if you do or damned if you don't"... if you use twisted.protocols.telnet it'll be removed and your code won't work, but if you use twisted.conch.telnet it could be changed and your code won't work.

That's life when you're depending on the volunteer efforts of others. :) That said, nothing is stopping you from using twisted.protocols.telnet: if it is someday removed, you can always distribute a copy of it with your software.

I'm not nearly concerned, BTW, about the <1.0 version number as I am about the "unstable" warning.

If twisted.conch is what people are supposed to be using, wouldn't it make sense to put that in the core install?

No, because the "core install" is a very special-purpose thing that most people aren't interested in. It's part of the "Twisted install" already, and that's what most people should install.

Sorry for being such a griper and for hammering a dead horse with this one. I guess I'd just like some assurance that my project, written using twisted.conch, will still work as the conch library gets upgraded to 1.0

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. To look at this problem another way, how would you feel if you started to use twisted.conch.telnet and found that it satisfied 90% of your use-case, but that due to one critical oversight, you couldn't supply the remaining 10% yourself but instead had to write your own telnet library from scratch, because we had given up the freedom of making changes to twisted.conch.telnet? That wouldn't be any more useful to you than the current situation. What you can do is use it and provide feedback on what it does and does not do well and work with us to improve it to the point where we _are_ confident that it is good enough to abandon the hope of making further incompatible changes to it later on.

(at which point, I presume, it would become stable?)

Maybe or maybe not. Generally we mark APIs as stable or unstable with finer granularity than this. It is unlikely that all of conch will become stable at the same time. Jean-Paul

Heh, I was typing my reply about the website issue just as you were typing that email. Please, please, please fix that! Not only is the link's label the exact OPPOSITE as the content of the link, but all the other links on that (and the other) pages just link to full-page versions of "Undocumented." There is no possible way I could have known there was documentation hidden in there. Please fix!

To look at this problem another way, how would you feel if you started to use twisted.conch.telnet and found that it satisfied 90% of your use-case, but that due to one critical oversight, you couldn't supply the remaining 10% yourself but instead had to write your own telnet library from scratch, because we had given up the freedom of making changes to twisted.conch.telnet? That wouldn't be any more useful to you than the current situation.

Well, at the moment, the 90% I need to know is "if there's a security hole in the library, and my users are forced to update it, will my code no longer work because the interfaces changed?" I understand that there is no warranty, yadda yadda, and I understand that it's a volunteer effort and open source yadda yadda, but what I really *need* right now is the guarantee of stability. If conch's interface can change at any time (on servers that I may not control and have no say over their installed software), it becomes impossible for me to support my program and I'd be better off writing my own, even if that is more code. So for me, stability is the number one concern. However, I can't distribute the old version of the library because of the potential for undiscovered security holes. So we're back to "damned if I do, damned if I don't." Anyway, I'll keep looking. Maybe there's another library out there that implements what I need with a stable interface. Thanks.

Michael Hudson wrote:

Do you have any concrete ideas for what to put in that box instead? "No class docstring, 3/14 method docstrings" or something? Seems a bit verbose.

Since there's nothing useful in those boxes right now, I'd suggest doing one (or both) of the following: (1) (easiest) leave those boxes off entirely, since the Interface/Class names are links to the docs anyway, or (2) add summary Interface / Class docstrings to the source. And maybe generate some boilerplate above such tables that says something to the effect of: "Follow the links for more detailed documentation." Steve

On Sun, 19 Mar 2006 11:51:23 -0800, James Schend <blakeyrat@gmail.com> wrote:

So for me, stability is the number one concern. However, I can't distribute the old version of the library because of the potential for undiscovered security holes.

Practically speaking, the potential for security holes in the code you're discussing is vanishingly small. If there is a hole, we will likely release a minor-version patch for people still using recent affected releases, especially if the interface has changed. I say this because there have almost never been holes in Twisted, and when there have been, they've been spotted so fast that a minor-version release was available soon after they were introduced. You could easily include a tiny telnet module with your library without being worried about security holes. twisted.protocols.telnet is 159 SLOC; twisted.conch.telnet is 572 SLOC. If you know enough to be this concerned about security issues, surely you know enough to perform an audit of that small amount of code. (Especially when half of it, in the latter case, is simply declaration of constants.)

So we're back to "damned if I do, damned if I don't."

Welcome to the internet ;-). At the core of this issue is a labor problem. If you care about the telnet module, you could offer to provide the necessary labor (or funds!) maintain it, which would accellerate conch's official replacement of twisted.protocols.telnet, simplifying this situation. As maintainer, you could also exercise some degree of control over changes to the telnet module which would avoid breaking your application. What you are asking for right now is a guarantee of stability from people who simply don't have the resources to provide it to you. If we could do that for free, we probably would, but it needs a dedicated maintainer, a clear plan for future development, some documentation, etc etc.

Anyway, I'll keep looking. Maybe there's another library out there that implements what I need with a stable interface. Thanks.

This is vanishingly unlikely. If you find a Twisted clone (with a proper telnet implementation!) which is open source, well maintained, and well documented, please let us know where it is.

On 3/19/06, James Schend <blakeyrat@gmail.com> wrote:

One of the reasons to use a library, from my perspective, is that the interfaces can remain constant while the internals can be patched to ensure there are no bugs/security problems. For instance, when libpng had that buffer overflow last year, that single library was updated to correct the problem without affecting the functioning of any of the hundreds of programs that used it. It would be nice if the same could be done with Twisted but, as there is not currently a stable interface to it, that is impossible.

You said a bunch of things about how great it is to share code, and then made a weird assertion at the end about how some code in Twisted can't provide those great things. I'm not seeing the logical connection. Why can't we fix bugs, even security holes, just because the API is marked "unstable"? The only thing an "unstable" marker means that we are allowing ourselves the freedom of *also* changing the interface of the library. Anyway, as Itamar said, we always try to keep interfaces backwards compatible, even if they're marked as "unstable". The "unstable" marker does not say "we will ignore backwards compatibility", it is "if there are really good reasons, we might might break the interface". The only reason the *old* telnet implementation wasn't marked unstable was that nobody had even gotten around to adding a "stability" marker to it. Anyway, as it comes down to it, you're telling us to do things that we can't do without funding or more developer time. Even if there are thousands of developers who feel the same way, that doesn't change anything. Regardless, I still strongy suggest that you're reading way too much into the "API stability: unstable" marker, and I encourage you to use and help improve the Telnet code in Conch. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/

Look, all I want to be able to do is to write a program that can sit on a server for 10 years and run without me touching it. My users won't necessarily be technical and (much more likely) the program won't necessarily be maintained, so it is important that the libraries used, if any, won't break my code with updates. Maybe those are strange, unheard-of, constraints, but either way it looks like I have to write my own telnet server. I'd rather not, and I'm sure you'd rather have me as a user, but that's just the way it is. One last question: How do I uninstall Twisted? Do I re-run setup.py with "setup.py core uninstall?" On 3/19/06, glyph@divmod.com <glyph@divmod.com> wrote:

On Sun, 19 Mar 2006 18:32:31 -0800, James Schend <blakeyrat@gmail.com> wrote:

...

Well, if this is a library that you expect people to actually be able to use in production software, I hope that you take what I say under some consideration.

Many of the people using Twisted in production, and there are many who are, contribute to its maintenance in some way. It is still a project actively under development. However, it's hardly a necessity, and probably half of the people on the Success Stories page have never contributed anything to Twisted.

It sure would be nice if it were totally stable and full-featured, but it would also be nice if I had a pony. There is a difference between not having the kind of total stability and commercial-grade support guarantees that you want (for free) and being unusable in production. Code which is *way* worse than Twisted, with less maintenance and security attention than Twisted, is used in "production" all over the world, every day.

By the way, there are definitely people who *will* provide commercial-grade support for Twisted, any version you like, for a very reasonable fee. My company (Divmod) is among them. Unfortunately for us, it turns out nobody actually needs commercial-grade support for Twisted, because the code actually stands on its own pretty well :).

...

One of the reasons to use a library, from my perspective, is that the interfaces can remain constant while the internals can be patched to ensure there are no bugs/security problems. For instance, when libpng had that buffer overflow last year, that single library was updated to correct the problem without affecting the functioning of any of the hundreds of programs that used it.

libpng's interface is hardly stable. There is a libpng, libpng2, and libpng3, and a planned libpng4. The interface changes every major release. They may not write "deprecated! unstable!" all over version 2, but I bet if you were to ask a libpng developer which version you should use to write new code, they wouldn't say "use the one we wrote five years ago and haven't been able to update except for security reasons".

Granted, libpng provides better support for side-by-side installation of mutually incompatible versions than Twisted does, however, it also doesn't provide any "soft" interface changes, deprecation warnings, etc. It just totally breaks the ABI every major release.

Now, you could easily talk about the merits of sustained parallel development tracks and side-by-side installation, but I don't think that this is a worthwhile discussion until someone is actually going to provide the resources to *do* it. The best you can do right now is discourage and annoy the people who are dealing with maintaining Twisted's *one* sustained track of development.

...

It would be nice if the same could be done with Twisted but, as there is not currently a stable interface to it, that is impossible.

This is just FUD. There is nothing magical about the word "stable" which makes security patches possible.

If there are critical security bugs in Twisted, it may well be in someone's interest to do a maintenance release of each release-branch which is used in production. There is nothing about having a "stable interface" which makes this easy or free.

...

I appreciate the work you put in to Twisted, but I'm sure I'm not the only developer who feels this way about it.

The sort of security problem you are concerned about is literally so rare that it has never happened before. Twisted's security bugs (2 in its 5-year history) have always been caught well before another major-version release has happened.

I'm sure other developers feel the same way you do, and that's unfortunate, but you will pardon me if I don't immediately re-configure Twisted Matrix Labs' priorities around an eventuality that has never happened and may only hypothetically happen in the future, simply because it concerns you.

...

At the bare minimum, you probably should deprecate an interface until there's something stable to replace it with. I wouldn't consider a library with *no* stable interface whatsoever "well maintained," personally.

http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=deprecated

deprecate \dep"re*cate\ (d[e^]p"r[-e]*k[=a]t), v. t. To pray against, as an evil; to seek to avert by prayer; to seek deliverance from; to express deep regret for; to desire the removal of.

When things are deprecated, it is a statement that we now understand that approach to be a bad idea, which we regret having implemented. They may not be deprecated in favor of anything, or they may be deprecated in favor of something that isn't a part of Twisted (for example, the components system was deprecated in favor of Zope Interface). Sometimes, a particular API is just trying to accomplish an impossible task, or provides a bad way to structure code. The first iteration of twisted.cred comes to mind. Nothing attempts to do what the first version of twisted.cred did; the new cred has a much more restricted goal, because what the first version of twisted.cred was trying to do was nearly impossible and it turned out that it didn't actually work.

There was nothing stable in the telnet code in the first place; it just wasn't properly annotated as such.

Something being "well-maintained" within Twisted means that there are people working on it and actively interested in improving and advancing it. To some extent, that is actually the *opposite* of stable; stable things are things that mostly work OK already. It is possible to be both, of course. Some stable things, like the reactor, have a stable interface but a known-to-be-sub-optimal implementation, and the maintainers' work is being put towards improving just the implementation. Those sorts of things are unfortunately rare, because interfaces turn out to be harder to get right than implementations.

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

James Schend wrote:

Look, all I want to be able to do is to write a program that can sit on a server for 10 years and run without me touching it. My users won't necessarily be technical and (much more likely) the program won't necessarily be maintained, so it is important that the libraries used, if any, won't break my code with updates.

As long as noone maintains it, the libraries won't get updated, so you have nothing to worry about. ;^) If you have a look at PEP 5 http://www.python.org/doc/peps/pep-0005/ you will notice that "There must be at least a one-year transition period between the release of the transitional version of Python and the release of the backwards incompatible version." In practice, it's typically longer than that, but don't bet on it. You never have anything near 10 year API stability guarantees in Python if you update the software. Even basic syntax and built in operators can change. Changes of the division operator have been announced more than a year ago, so you already live on borrowed time if you don't use "from __future__ import division" and make sure that no libraries you use rely on the old behaviour. There is no way Twisted can shield you from these Python problems. It's true that a lot of Python code written 10 years ago still work, but you have poor timing if you want to project that forward. There will most probably be core language changes in that time that will require maintenance work, such as the default behaviour of the division operator, removal of old style classes, enforcement of new style class exceptions, maybe even removal of some statements etc. Don't expect any support for Python 2.x in ten years. I'm sure you can still run a Python/Twisted app written today in ten years from now, but don't expect to do it on an updated platform. If this worries you, don't use Python.

On 20 Mar 2006, at 06:42, James Schend wrote:

The link that is oh-so-helpfully labelled Undocumented:

http://schend.net/images/screenshots/twisted/no_documentation.png

Actually links to a page full of extensive documentation:

http://schend.net/images/screenshots/twisted/documentation.png

I guess this should be filed as a bug against the new documentation system: "classes with no docstring should be labelled 'no class docstring' rather than 'undocumented'" or something like that.