Re: [Twisted-Python] schemas (in Zope and Foolscap)

Brian Warner wrote:

[my apologies if this gets duplicated, I've been having SMTP problems]

It appears that this message was relayed twice, once today and once yesterday, and both times I only received the cc to me, and not the copy that was addressed to the twisted-python list. So I'm quoting the whole message below in case others haven't seen it.

Stephen Waterbury writes:

...

I am in the process of converting some of my code to use zope.schema instead of a similar thing that I independently invented. Since I am also planning on using Foolscap, and since Foolscap has a schema module, I compared zope.schema and foolscap.schema, and to me they seem to have much overlap in design intent. In particular, in the zope.schema use cases ...

...

So I was wondering -- would it make sense to use zope.schema in Foolscap?

Yeah, that sounds quite intriguing. The Foolscap schemas serve a number of similar purposes:

#1: provide a clear place to document a program's remote interfaces. This role is aimed at humans, not programs.

#2: make it convenient to enforce those interfaces on the receiving end.. by relying upon the guards, the code that actually does stuff is easier to read and easier to analyze for other potential problems

#3: enforce those interfaces on the sending end, to bring the discovery of programming errors closer to their cause

#4: enforce those interfaces at the receiving *wire*, to mitigate resource-exhaustion attacks

#1 is satisfied by pretty much anything, as it's more a set of documentation #conventions than anything else. The existing zope.interface.Interface class #does this pretty well.

The rest require that the interfaces be machine-parseable. I enhanced z.i's Interface (in foolscap.RemoteInterface) to allow the arguments and return values to have meaning.. if zope.schema does something similar, I'd love to take a look at it and see if it can capture the same expressiveness.

Yes, that appears to be one of the things zope.schema does.

I'm undecided as to whether #4 is a good idea or not. It seemed like a good idea when I first started, but I've had some smart people tell me it's not the best place to attempt to solve DoS problems. Worse yet, the implementation is so incomplete that I've personally had to disable schema checking on things that would otherwise be useful (in particular I think a "PolyConstraint" in which the two branches are both containers fails to match the tokens on the wire correctly, even though such a thing would be quite useful just at the post-serialization phase). So I'm tempted to drop #4 as a design criterion and if z.s can represent enough to make that work, great, if not, drop the feature.

...

A nice practical factor is that zope.schema has been packaged as a separate namespace module, similarly to zope.interface.

I have to admit that I'm slightly hesitant to add a new dependency. But maybe it isn't too big or too unwieldy. There doesn't seem to be a debian package for it, though..

Right, and for both of those reasons (extra dependency and the debian packaging lag) I would like to see zope.schema merged into zope.interface, since zope.schema depends on zope.interface and I don't think it would be a big burden on zope.interface to carry along zope.schema even if zope.interface users don't all use it. I'll do a little lobbying for that on interface-dev (which has been extremely quiet lately ... probably a good sign).

But I'll definitely check it out. Thanks for the suggestion!

-Brian