[Web-SIG] A Python Web Application Package and Format

[Eric Larson]

Hi, 


On Apr 14, 2011, at 4:02 AM, Graham Dumpleton wrote:

> On 14 April 2011 18:22, Alice Bevan–McGregor <alice at gothcandy.com> wrote:
>> Howdy!
>> 
>> I suspect you're thinking a little too low-level.
> 
> Exactly, I am trying to walk before running. Things always fall down
> here because people try and take too large a leap rather than an
> incremental approach, solving one small problem at a time.
> 
> Thus please don't think that because I am replying to your message
> that I am specifically commenting about your plans. See this as a side
> comment and don't try and evaluate it only in the context of your
> ideas.
> 
>> On 2011-04-14 00:53:09 -0700, Graham Dumpleton said:
>> 
>>> On 14 April 2011 16:57, Alice Bevan–McGregor <alice at gothcandy.com> wrote:
>>>>> 
>>>>> 3. Define how to get the WSGI app.  This is WSGI specific, but (1) is
>>>>> *not* WSGI specific (it's only Python specific, and would apply well to
>>>>> other platforms)
>>>> 
>>>> I could imagine there would be multiple "application types":
>>>> 
>>>> :: WSGI application.  Define a package dot-notation entry point to a WSGI
>>>> application factory.
>>> 
>>> Why can't it be a path to a WSGI script file?
>> 
>> No reason it couldn't be.
>> 
>> app.type = wsgi
>> app.target = /myapp.wsgi:application
>> 
>> (Paths relative to the folder the application is installed into, and dots
>> after a slash are filename parts, not module separators.)
>> 
>> But then, how do you configure it?  Using a factory (which is passed the
>> from-appserver configuration) makes a lot of sense.
>> 
>>> This actually works more universally as it works for servers which map
>>> URLs to file based
>>> resources as well.
>> 
>> First, .wsgi files (after a few quick Google searches) are only used by
>> mod_wsgi.  I wouldn't call that "universal", unless you can point out the
>> other major web servers that support that format.
> 
> The WGSI module for nginx used them, as does uWSGI and either one of
> Phusion Passenger or new Mongrel WSGI support rely on a script file.
> 
> You also have CGI, FASTCGI, SCGI and AJP also using script files.
> 
> Don't get hung up on the extension of .wsgi, it is the concept of a
> script file which is stored in the file system in an arbitrary
> location to which a URL maps.
> 
>> You'll have to describe the "map URLs to file based resources" issue, since
>> every web server I've ever encountered (Apache, Nginx, Lighttpd, etc.) works
>> that way.
> 
> Which supports what I am saying, but you for some reason decided to
> focus on '.wsgi' as an extension which wasn't the point.
> 
>> Only if someone is willing to get really hokey with the system
>> described thus far would any application-scope web servers be running.
> 
> Forget for a moment trying to tie this to your larger designs and see
> it as more of a basic underlying concept. Ie., the baby step before
> you try and run.
> 
>>> Also allows alternate extensions than .py and also allows basename of file
>>> name to be arbitrarily named, both of which help with those same servers
>>> which map URLs to file base resources.
>> 
>> Again, you'll have to elaborate or at least point to some existing
>> documentation on this.
>> 
>> I've never encountered a problem with that, nor do any of my scripts end in
>> .py.
> 
> Lack of an extension is fine if you have configured Apache with a
> dedicated cgi-bin or fastcgi-bin directory where an extension is
> irrelevant because you have:
> 
>  SetHandler cgi-script
> 
> But many Apache server configurations use:
> 
>  AddHandler cgi-script .py
> 
> Ie., handler dispatch is based off extension, the .py extension quite
> often being associated with CGI script execution.
> 
> You often see:
> 
>  AddHandler fcgid-script .fcgid
> 
> Which says certain resource is to be started up as FASTCGI process.
> 
> For both these it expects those scripts to be self contained programs
> which fire up the mechanics of interfacing with CGI or FASTCGI
> protocols.
> 
> This means that you usually have to stick that boilerplate at the end
> of the script.
> 
> This is where though FASTCGI deployment usually sucks bad. This is
> because it is put on the user to get the boilerplate and remainder of
> WSGI script perfect from the outset. If you don't, because FASTCGI
> technically doesn't allow for stdout/stderr at point of startup, if
> there is an error on import it is lost and user has no idea. So many
> times you see people winging about setting up stuff on the likes of
> DreamHost because of FASTCGI being a pain like this.
> 
> In the PHP world they don't have to deal with this boilerplate
> nonsense. Instead there is a PHP launcher script associated with
> FASTCGI module. So you have:
> 
>  AddHandler fcgid-script .php
> 
> but also a mapping in FASTCGI module configuration that says rather
> than execute .php script if runs the launcher script instead. That way
> the launcher script can get everything setup properly to then call
> into the script.
> 
> Nothing exists for Python like that, but if you did then it makes no
> sense to use .py because of the mapping that extension often already
> has in Apache. In that case you would have .wsgi script file mapped to
> FASTCGI but FASTCGI configured to run a WSGI launcher. That launcher
> script would setup stdout/stderr, ensure flup is loaded properly and
> only then load the WSGI script file and execute. This way the system
> administrators could ensure the launcher is working and users only
> have to worry about dumping a WSGI script file with right extension in
> a directory and it will work without all the pain. Also allows the
> system admins to properly control number of processes/threads whereas
> at present users can override what system admins would like to
> restrict them to.
> 
> So, a concept of a script file simply works better with Apache and to
> some degree other servers. This is because of how such servers
> determine what handler to use from the extension
> 
> As to the file name, you can't stop people using arbitrary stuff in
> file names, ie., dashes as a prime example. So when using servers
> which map URLs to file system resources you have to deal with it.
> 
>>> It also allows same name WSGI script file to exist in multiple locations
>>> managed by same server without having to create an overarching package
>>> structure with __init__.py files everywhere.
>> 
>> Packages aren't a bad thing.  In fact, as described so far, a top level
>> package is required.
> 
> You are thinking ahead to your bigger ideas. That isn't what I am
> talking about. You can't when using a web server which can map URLs to
> resources within a hierarchical directory structure have that
> structure be a package with __init__.py files in directories, it just
> doesn't work as all the scripts could be totally unrelated and not
> part of one application.
> 

I want to give a big +1 for Graham's suggestion. Using a script is a great way to make the communication between the larger system and the application trivial. The larger system needs to know how to run the app. If it is a script then you just run the script. There should still be some information regarding apache/nginx config if necessary, but basing that on the expectation there is a single script is a better approach than presuming a config can provide enough information to eventually create some script that apache/nginx/etc. might need to use. 

Eric

> Graham
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: http://mail.python.org/mailman/options/web-sig/eric%40ionrock.org