“Mix and match Web components with Python WSGI”

Subtitle: Learn about the Python standard for building Web applications with maximum flexibility
Synopsis: Learn to create and reuse components in your Web server using Python. The Python community created the Web Server Gateway Interface (WSGI), a standard for creating Python Web components that work across servers and frameworks. It provides a way to develop Web applications that take advantage of the many strengths of different Web tools. This article introduces WSGI and shows how to develop components that contribute to well-designed Web applications.

Despite the ripples in the Python community over Guido's endorsement of Django (more on this in a later posting), I'm not the least bit interested in any one Python Web framework any more. WSGI has set me free. WSGI is brilliant. It's certainly flawed, largely because of legacy requirements, but the fact that it's so good despite those flaws is amazing.

I wrote this article because I think too many introductions to WSGI, and especially middleware, are either too simple or too complicated. In line with my usual article writing philosophy of what could I have read when I started out to make me understand this topic more clearly, I've tried to provide sharp illustration of the WSGI model, and a few clear and practical examples. The articles I read that were too simple glossed over nuances that I think should really be grasped from the beginning (and are not that intimidating). In the too-complicated corner is primarily PEP 333 itself, which is fairly well written, but too rigorous for an intro. In addition, I think the example of WSGI middleware in the PEP is very poor. I'm quite proud of the example I crafted for this article, and I hope it helps encourage more people to create middleware.

I do want to put in a good word for Ian Bicking and Paste. He has put in tireless effort to evangelize WSGI (it was his patient discussion that won me over to WSGI). In his Paste toolkit, he's turned WSGI's theoretical strengths into readily-available code. On the first project I undertook using a Paste-based framework (Pylons), I was amazed at my productivity, even considering that I'm used to productive programming in Python. The experience certainly left me wondering why, BDFL or no BDFL, I would choose a huge mega-framework over a loosely-coupled system of rich components.

[Uche Ogbuji]

via Copia

6 responses

Hi Uche. I'm afraid your middleware example in that article is not WSGI-compliant, because it doesn't support the wrapped application having a "close()" method:

"""If the iterable returned by the application has a close() method, the server or gateway must call that method upon completion of the current request, whether the request was completed normally, or terminated early due to an error. (This is to support resource release by the application. This protocol is intended to complement PEP 325's generator support, and other common iterables with close() methods."""

I hope you'll update the article so as to not lead others astray in this respect. Thanks.

— Phillip J. Eby

PJE,

First of all, if the article needs change, I can have that arranged fairly quickly.

I am puzzled about one thing. When I do add close() support to safexhtml.py, wsgiref 0.1.2 is *not* calling it. Am I missing something? I'll e-mail you my updated files...

--Uche

— Uche

That's because close() is an attribute of the application return value, not the application object itself. Please see the example middleware in the PEP for how to do this correctly, as for this kind of middleware, you have to have a separate iterator object in order to properly support closing.

In Python 2.5, you can do this using try/finally with a generator, because generators have a close() method and the generator's finally clauses will be run when close() is called. So, if you want a quick way to make your code compatible, you can do that as long as you require Python 2.5 and explain the issue so people know they have to follow the PEP middleware example's approach if they need to work with older Python versions.

Uche,

Well you know how careful I am with WSGI and with this in mind you should better understand my comment.

I think your article fails at the same point every other single article I've read about WSGI.

It's a nice Hello World (a very well explained one though) showing how to create one middleware and how to wrap any WSGI application with it but it is not a real worl example of an application with multiple requirements per path. WSGI applies itself to an application object (whether the application or the middleware acting as such), if you need distinct behaviors per path you will have to make some unfriendly if/elif/else statements in your middleware.

For instance, say two URIs lead to two resources in your application. On a POST to those URIs you want to validate the Content-Type of the Request in each case for distinct CT. Unless I am missing something (and if I do please let me know and I will be happy to change my mind) you are forced within a middleware to hack something to test the request Content-Type against those URIs. In the same vein, how do I enable several distinct dispatchers within a WSGI application? Again by an ugly hack within a middleware I suppose based on the request URI.

What bothers me about WSGI is not the specification which is actually well written and has very sensible ideas, it's the propaganda that WSGI will save your day.

It's true in some cases and it's not in some others. WSGI is not a God send. It's just a different approach to web development. A very clever one indeed but not relevant in every situations.

When I see:

"""

Warning

If you remove any of the default middleware you are likely to find that various parts of Pylons stop working!

"""

In the Pylons documentation, I can understand the requirement from a framework point of view to force a stack of middlewares not to be modified by changing one of them, but then let's stop the "with WSGI you can plug and play". This is simply not true in many cases.

Even your example which says "another reason to use the environment is to communicate down the WSGI stack the content has been modified" shows how you can build a non loosely coupled application on top of so-called loosely coupled components. If I build an application with your middleware and my application relies on the 'safexhtml.active' environ value, what does happen if a deployer changes the middleware for one that does exactly the same job but which doesn't set such environ variable?

I am not picky, I am realistic. During the development of an application you will have to strongly rely on specific middleware that cannot be moved out.

This is perfectly fine and I do accept it but I find it quite sad that the WSGI defenders don't say it out loud.

So I will state it again, I think WSGI can be extremly helpful in many cases, and in those cases I will agree they certainly increase your productivity, but there are cases where WSGI is just not the right tool and other designs can be much more powerful. Exactly like CherryPy is fantastic but not always the best tool for the job. There is no such thing.

The day I will start seeing more of those statements I will stop my rant.

That being said Uche, I think your article is one of the best about introducing WSGI I have read and I agree it would have helped me a lot when I started with WSGI. You have reached your goal IMO :)

I do believe you won't agree with me Uche and that's fine. I hope you will understand that I am not against WSGI per se, I am however tired of the lack of clarity about the shortcomings of WSGI.

- Sylvain

— Sylvain Hellegouarch

Sylvain,

I don't really understand you very well, but then again I don't think I've ever really understood your problem with WSGI.

First of all, there is nothing wrong with if/then/else, is there? It's a cornerstone of structured programming. In cases where if/then/else can be clumsy I don't see how WSGI prevents you from using other dispatch mechanisms such as polymorphism.

And I don't understand your Content-Type example. Could you elaborate? Perhaps with a code example? Show something you can do with, say CGI or CherryPy that you can't do elegantly with WSGI. It's Ok to post to your Weblog. I read that :-)

Also, what's so strange about the idea that a framework might depend on a particular bit of WSGI middleware? Almost all software has some dependency. Why can't Pylons have some dependency? CherryPy certainly has dependencies. WHy is it important for "WSGI defenders" to make the obvious statement that "software often has dependencies"?

I'd appreciate any clarification.

Sylvain,

The I think the appeal of the middleware that Uche created (and WSGI middleware in general) is that it is a component that can be "mixed and matched" into other WSGI frameworks immediately. His tool is available for use within CherryPy, Pylons, web.py and any other present/future WSGI compatible framework.

There is no need to write a Pylons version, a CherryPy version, etc. Because they adhere to the WSGI spec, and so does Uche's middleware (at least it is close enough for horseshoes and hand grenades ;-) ), they can work together quite simply.

Like Uche, I don't see the problem with having dependencies on middleware. But what if one/more of Pylons' dependencies were also useful to Turbogears and they integrated it/them? Wouldn't that be handy?

Anyhow, I think you might be expecting more from WSGI than it is capable of, which from what I gather is <srong>connecting Python web components</strong>. It is a good step towards making all the work that all of us are doing in Python web development more useful to each other. Pretty cool if you ask me :-)

We'll chat more about it someday I imagine ;-)

Christian

— Christian