Hi, I just uploaded CorePost 0.0.5 to PyPi. CorePost is a REST microframework built on top of the core twisted.web APIs, providing a Flask-inspired API for building REST applications. Version 0.0.5 integrates FormEncode for integrated path/form/query argument validation, similar to the way Pylons/TurboGears use it. Example: from corepost.web import CorePost, validate from corepost.enums import Http from formencode import Schema, validators app = CorePost() class TestSchema(Schema): allow_extra_fields = True childId = validators.Regex(regex="^value1|value2$") @app.route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs) @app.route("/validate/<int:rootId>/custom",Http.POST) @validate(childId=validators.Regex(regex="^value1|value2$")) def postValidateCustom(request,rootId,childId,**kwargs): '''Validate using argument-specific validators' return "%s - %s - %s" % (rootId,childId,kwargs) More docs on our site: https://github.com/jacek99/corepost Any feedback is welcome Cheers, Jacek

The point is that I more or less blindly copied the Flask API when starting on this, due to relative lack of experience with Python :-) I figured their way must be the Pythonic way and went along with it. I like your suggestion a lot more, it is definitely more OOP and cleaner. Expect it to be reworked accordingly in the next release Cheers Jacek On Sun, Sep 4, 2011 at 12:01 AM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:

On Sep 3, 2011, at 8:28 PM, Jacek Furmankiewicz wrote:

Any feedback is welcome

Hi Jacek,

Great to see more development going into Twisted-based web stuff! :)

However, I do have one question. Maybe I'm missing something about the way Flask does things, but it seems very odd to me that the decorators you're using are applied to global functions, rather than instances of an object. For example, instead of:

app = CorePost() ... @app.route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

You could do:

class MyPost(CorePost): @route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(self,request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

This would allow for re-usable objects; for example, rather than having a "blog article create" API (sorry for the uninspired example, it's late) for your entire site, you would have a "article create" API on a "Blog", which would enable you to have multiple Blog objects (perhaps with different authors, in different permission domains, etc). This would also make re-using the relevant objects between different applications easier.

In other words, global variables are bad, and this looks like it depends rather heavily on them.

Any thoughts on this? Am I missing the point?

Thanks,

-glyph

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

D'oh. Finally found how to do it. I can use the decorator to attach additional route-specific attributes to the function and then the class instance can scan for it from the constructor, auto-registering each of its own methods that have route info attached to them. Sorry for previous question Jacek On Mon, Sep 5, 2011 at 12:14 PM, Jacek Furmankiewicz <jacek99@gmail.com>wrote:

Hi Glyph,

I looked at your suggestion, but unfortunately the implementation is very complex, if not impossible.

The main problem is that a) a class method with a decorator "forgets" its class, so it's impossible from the decorator which class it belongs to. The function has not been bound to a class yet when the decorator is called for the first time, so there is no way for it to notify the containing class that this function defines a route for it

b) is is next to impossible for a class to scan its own function and find their decorators. I've seen some hacks on StackOverflow where it actually parses the source code, but that is an ugly hack to say the least (and probably prone to many bugs)

In general, it seems decorators on class methods are missing such crucial functionality as finding out which class the method belongs to. Sort of a key requirement, if you ask me (at least after lots of experience with Java or .Net reflection, where getting this sort of info is trivial).

if you have any suggestions on how to accomplish your recommendation, I would greatly appreciate it.

The decorator in question that I would need to take out of the CorePost class and make it a standalone function looks like this:

def route(self,url,methods=(Http.GET,),accepts=MediaType.WILDCARD,produces=None,cache=True): """Main decorator for registering REST functions """ def wrap(f,*args,**kwargs): self.__registerFunction(f, url, methods, accepts, produces,cache) return f return wrap

it's obtaining the reference to 'self' when it is not a class method any more is the problem. Not sure how to get around it.

Cheers, Jacek

On Sun, Sep 4, 2011 at 12:01 AM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:

...

On Sep 3, 2011, at 8:28 PM, Jacek Furmankiewicz wrote:

Any feedback is welcome

Hi Jacek,

Great to see more development going into Twisted-based web stuff! :)

However, I do have one question. Maybe I'm missing something about the way Flask does things, but it seems very odd to me that the decorators you're using are applied to global functions, rather than instances of an object. For example, instead of:

app = CorePost() ... @app.route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

You could do:

class MyPost(CorePost): @route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(self,request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

This would allow for re-usable objects; for example, rather than having a "blog article create" API (sorry for the uninspired example, it's late) for your entire site, you would have a "article create" API on a "Blog", which would enable you to have multiple Blog objects (perhaps with different authors, in different permission domains, etc). This would also make re-using the relevant objects between different applications easier.

In other words, global variables are bad, and this looks like it depends rather heavily on them.

Any thoughts on this? Am I missing the point?

Thanks,

-glyph

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

Actually, it turns out our solution was a little more complicated :P Here is our basic implementation (our actual class is a bit more involved and allows multiple routes per method, regular expressions, etc..). The view's method decorators are evaluated before the view's class decorator. The class decorator looks at all the methods and sees which have been decorated with the route. class Routing(object): routes = {} @classmethod def register(cls, viewclass): # Check a class for any methods that have routes as attributes # and add these to the routing dictionary for k,v in viewclass.__dict__.items(): if hasattr(v, 'route'): print "Registering %s method %s with route %s"%(viewclass, v.__name__, v.route) cls.routes[v.route] = (viewclass, v.__name__) return cls @classmethod def addroute(cls, route): # This decorator adds the specified routes as attributes to the methods. # These routes will be found when the view class is registered. def wrap(handler): handler.route = route return handler return wrap @classmethod def resolve(cls, route): for r in cls.routes: if r == route: print "Found handler for %s:"%route, cls.routes[r] view, method = cls.routes[r] inst = view() return getattr(inst, method) @Routing.register class View(object): @Routing.addroute(r'/blog/post/') def dosomething(self, title=None, body=None): print "Adding blog post:", title, body handler = Routing.resolve('/blog/post/') handler(title="Test", body="ok") Thanks, Ian On Sep 5, 2011, at 11:14 AM, Jacek Furmankiewicz wrote:

Hi Glyph,

I looked at your suggestion, but unfortunately the implementation is very complex, if not impossible.

The main problem is that a) a class method with a decorator "forgets" its class, so it's impossible from the decorator which class it belongs to. The function has not been bound to a class yet when the decorator is called for the first time, so there is no way for it to notify the containing class that this function defines a route for it

b) is is next to impossible for a class to scan its own function and find their decorators. I've seen some hacks on StackOverflow where it actually parses the source code, but that is an ugly hack to say the least (and probably prone to many bugs)

In general, it seems decorators on class methods are missing such crucial functionality as finding out which class the method belongs to. Sort of a key requirement, if you ask me (at least after lots of experience with Java or .Net reflection, where getting this sort of info is trivial).

if you have any suggestions on how to accomplish your recommendation, I would greatly appreciate it.

The decorator in question that I would need to take out of the CorePost class and make it a standalone function looks like this:

def route(self,url,methods=(Http.GET,),accepts=MediaType.WILDCARD,produces=None,cache=True): """Main decorator for registering REST functions """ def wrap(f,*args,**kwargs): self.__registerFunction(f, url, methods, accepts, produces,cache) return f return wrap

it's obtaining the reference to 'self' when it is not a class method any more is the problem. Not sure how to get around it.

Cheers, Jacek

On Sun, Sep 4, 2011 at 12:01 AM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

On Sep 3, 2011, at 8:28 PM, Jacek Furmankiewicz wrote:

...

Any feedback is welcome

Hi Jacek,

Great to see more development going into Twisted-based web stuff! :)

However, I do have one question. Maybe I'm missing something about the way Flask does things, but it seems very odd to me that the decorators you're using are applied to global functions, rather than instances of an object. For example, instead of:

app = CorePost() ... @app.route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

You could do:

class MyPost(CorePost): @route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(self,request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

This would allow for re-usable objects; for example, rather than having a "blog article create" API (sorry for the uninspired example, it's late) for your entire site, you would have a "article create" API on a "Blog", which would enable you to have multiple Blog objects (perhaps with different authors, in different permission domains, etc). This would also make re-using the relevant objects between different applications easier.

In other words, global variables are bad, and this looks like it depends rather heavily on them.

Any thoughts on this? Am I missing the point?

Thanks,

-glyph

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

And the re-designed version 0.0.6 is out on PyPI. More detailed docs on github: https://github.com/jacek99/corepost Cheers, Jacek On Mon, Sep 5, 2011 at 1:41 PM, Jacek Furmankiewicz <jacek99@gmail.com>wrote:

Hi Ian, I think the problem though is that your solution relies on static class-level methods and variables, whereas I need more instance-level.

Here's my solution (seems to be working so far)

def route(url,methods=(Http.GET,),accepts=MediaType.WILDCARD,produces=None,cache=True): ''' Main decorator for registering REST functions ''' def decorator(f): def wrap(*args,**kwargs): return f router = RequestRouter(f, url, methods, accepts, produces, cache) setattr(wrap,'corepostRequestRouter',router)

return wrap return decorator

and then the following method is called from the instance constructor:

def __registerRouters(self): from types import FunctionType for _,func in self.__class__.__dict__.iteritems(): if type(func) == FunctionType and hasattr(func,'corepostRequestRouter'): rq = func.corepostRequestRouter for method in rq.methods: self.__urls[method][rq.url] = rq self.__routers[func] = rq # needed so that we can lookup the router for a specific function

I am updating docs and examples, 0.0.6 should be out soon with these changes, here's what the API looks like now with these changes:

class HomeApp(CorePost):

@route("/") def home_root(self,request,**kwargs): return "HOME %s" % kwargs

class Module1(CorePost):

@route("/",Http.GET) def module1_get(self,request,**kwargs): return request.path

@route("/sub",Http.GET) def module1e_sub(self,request,**kwargs): return request.path

class Module2(CorePost):

@route("/",Http.GET) def module2_get(self,request,**kwargs): return request.path

@route("/sub",Http.GET) def module2_sub(self,request,**kwargs): return request.path

def run_app_multi(): app = Resource() app.putChild('', HomeApp()) app.putChild('module1',Module1()) app.putChild('module2',Module2())

factory = Site(app) reactor.listenTCP(8081, factory) reactor.run()

Jacek

On Mon, Sep 5, 2011 at 1:25 PM, Ian Rees <ian@ianrees.net> wrote:

...

Actually, it turns out our solution was a little more complicated :P Here is our basic implementation (our actual class is a bit more involved and allows multiple routes per method, regular expressions, etc..). The view's method decorators are evaluated before the view's class decorator. The class decorator looks at all the methods and sees which have been decorated with the route.

class Routing(object): routes = {}

@classmethod def register(cls, viewclass): # Check a class for any methods that have routes as attributes # and add these to the routing dictionary for k,v in viewclass.__dict__.items(): if hasattr(v, 'route'): print "Registering %s method %s with route %s"%(viewclass, v.__name__, v.route) cls.routes[v.route] = (viewclass, v.__name__) return cls

@classmethod def addroute(cls, route): # This decorator adds the specified routes as attributes to the methods. # These routes will be found when the view class is registered. def wrap(handler): handler.route = route return handler return wrap

@classmethod def resolve(cls, route): for r in cls.routes: if r == route: print "Found handler for %s:"%route, cls.routes[r] view, method = cls.routes[r] inst = view() return getattr(inst, method)

@Routing.register class View(object): @Routing.addroute(r'/blog/post/') def dosomething(self, title=None, body=None): print "Adding blog post:", title, body

handler = Routing.resolve('/blog/post/') handler(title="Test", body="ok")

Thanks, Ian

On Sep 5, 2011, at 11:14 AM, Jacek Furmankiewicz wrote:

...

Hi Glyph,

I looked at your suggestion, but unfortunately the implementation is very complex, if not impossible.

The main problem is that a) a class method with a decorator "forgets" its class, so it's impossible from the decorator which class it belongs to. The function has not been bound to a class yet when the decorator is called for the first time, so there is no way for it to notify the containing class that this function defines a route for it

b) is is next to impossible for a class to scan its own function and find their decorators. I've seen some hacks on StackOverflow where it actually parses the source code, but that is an ugly hack to say the least (and probably prone to many bugs)

In general, it seems decorators on class methods are missing such crucial functionality as finding out which class the method belongs to. Sort of a key requirement, if you ask me (at least after lots of experience with Java or .Net reflection, where getting this sort of info is trivial).

if you have any suggestions on how to accomplish your recommendation, I would greatly appreciate it.

The decorator in question that I would need to take out of the CorePost class and make it a standalone function looks like this:

def route(self,url,methods=(Http.GET,),accepts=MediaType.WILDCARD,produces=None,cache=True): """Main decorator for registering REST functions """ def wrap(f,*args,**kwargs): self.__registerFunction(f, url, methods, accepts, produces,cache) return f return wrap

it's obtaining the reference to 'self' when it is not a class method any more is the problem. Not sure how to get around it.

Cheers, Jacek

On Sun, Sep 4, 2011 at 12:01 AM, Glyph Lefkowitz < glyph@twistedmatrix.com> wrote:

On Sep 3, 2011, at 8:28 PM, Jacek Furmankiewicz wrote:

...

Any feedback is welcome

Hi Jacek,

Great to see more development going into Twisted-based web stuff! :)

However, I do have one question. Maybe I'm missing something about the way Flask does things, but it seems very odd to me that the decorators you're using are applied to global functions, rather than instances of an object. For example, instead of:

app = CorePost() ... @app.route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

You could do:

class MyPost(CorePost): @route("/validate/<int:rootId>/schema",Http.POST) @validate(schema=TestSchema) def postValidateSchema(self,request,rootId,childId,**kwargs): '''Validate using a common schema''' return "%s - %s - %s" % (rootId,childId,kwargs)

This would allow for re-usable objects; for example, rather than having a "blog article create" API (sorry for the uninspired example, it's late) for your entire site, you would have a "article create" API on a "Blog", which would enable you to have multiple Blog objects (perhaps with different authors, in different permission domains, etc). This would also make re-using the relevant objects between different applications easier.

In other words, global variables are bad, and this looks like it depends rather heavily on them.

Any thoughts on this? Am I missing the point?

Thanks,

-glyph

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

Fortunately enough the code change was minor. The biggest challenge was getting the decorator function and the class instance to communicate somehow...past that point it was simple. I would rather focus on my current problem (beside CorePost enhancements) of Twisted not installing under PyPy. PyPy does wonders for Twisted. Some of the issues are documented here: http://stackoverflow.com/questions/5126039/how-to-install-twisted-10-2-0-wit... The solution seems pretty trivial (see last comment). Getting Twisted to install under PyPy would be a major boom. Right now all the PyPy guys have to do PYTHONPATH hacks. My initial perf tests with PyPy/Twisted were the reason I chose Twisted over let's say Node.js. I just like Python more and the inline callback support is brilliant. Is there a Twisted bug for the PyPy installation issue? Jacek On Mon, Sep 5, 2011 at 7:09 PM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:

On Sep 5, 2011, at 10:46 AM, Jacek Furmankiewicz wrote:

And the re-designed version 0.0.6 is out on PyPI.

I'm glad I made that suggestion. That kind of turnaround is pretty impressive! Any way I can convince you to fix Twisted bugs that fast? (Since you're doing HTTP POSTs, <http://twistedmatrix.com/trac/ticket/288>, perhaps?)

Congrats on the new version,

-glyph

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

Do you guys have any ETA on this? The discussion within the ticket makes it seem much more complex than the simple fix someone suggested on StackOverflow. It would allow to pit Twisted directly against node.js (and others) and not suffer from CPython perf limitations. There is a lot of complaints in the node.js blogs about the type of spaghetti code async nested callbacks create... ...a chance to show how elegantly Twisted inline callback support solves this. Jacek On Tue, Sep 6, 2011 at 9:44 AM, Thijs Triemstra <lists@collab.nl> wrote:

On 09/06/2011 03:28 PM, Jacek Furmankiewicz wrote:

...

Fortunately enough the code change was minor. The biggest challenge was getting the decorator function and the class instance to communicate somehow...past that point it was simple.

I would rather focus on my current problem (beside CorePost enhancements) of Twisted not installing under PyPy. PyPy does wonders for Twisted.

Some of the issues are documented here:

http://stackoverflow.com/questions/5126039/how-to-install-twisted-10-2-0-wit...

...

The solution seems pretty trivial (see last comment).

Getting Twisted to install under PyPy would be a major boom. Right now all the PyPy guys have to do PYTHONPATH hacks.

My initial perf tests with PyPy/Twisted were the reason I chose Twisted over let's say Node.js. I just like Python more and the inline callback support is brilliant.

Is there a Twisted bug for the PyPy installation issue?

Jacek

That's ticket #5158: http://twistedmatrix.com/trac/ticket/5158

Cheers,

Thijs

_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web