I have written an e-Commerce shop in Django. It took three days, much of which was spent writing the backend – basket, order processing and so on. I am absolutely certain that I couldn't have done that with Satchmo. The first reason is that Satchmo isn't orthogonal; it's impossible to use Satchmo's basket/ordering system without using Satchmo's products front end. Front ends are very easy to write in Django and I needed something custom for this shop. Custom front ends are very hard to write in Satchmo – they involve writing custom extensions to the product model. The second reason is that Satchmo doesn't use Enterprise Design Patterns like Money.

Satchmo also provides code that is very much outside its remit as a framework, like this strange configuration property which embeds Google Analytics. Google Analytics code is trivial to paste into the templates should I want to do so. Why would I need this built into the platform?

Satchmo doesn't satisfy as a framework; it's more of an off-the-shelf application regardless of what the authors claim. Now I have Django web shop code, I won't be needing Satchmo.

• Expiring static files from the webserver. Some data can be cleaned up whenever a page is requested. On occasion, though, the application establishes the contract that a file will stay around for a fixed period of time. When access to these files is provided by the webserver (not through the application itself) then the files need to be deleted at a given future moment.
• Time-based notifications. For example, if you deal with dates and times in your web application it's sometimes necessary to actually notify users (most often, via email) at a given time. It's clearly not acceptable to wait until someone hits a page (possibly hours or days later) to issue these notifications.
• Syndication. Polling data on remote servers has to be done ready for when a user hits a page, because otherwise it can introduce an unacceptable delay while variously contactable remote hosts are queried.

In several of my web applications now I've come to a sticking point when it comes to scheduling events. As far as I know this is always left up to the developer to arrange. Scheduling events is considered outside their remit.

There are a few solutions I know of.

• The application can provide a script which the administrator must schedule to be run periodically at install time. Drupal, for example, recommends adding a crontab entry which periodically wgets a script on the web site. In redistributable apps, many users will obliviously skip this step and wonder why the application won't work.
• Run scheduled tasks after serving each page. This approach doesn't solve the above problems. In mod_php/perl/python applications this hogs a webserver thread too, which could degrade performance.
• There are websites like webcron.org that will fetch a script on your server at intervals. It would be madness to rely on this in your own applications or suggest this as a solution for a redistributable applications, so it's only suitable as a fallback if all else fails.
• The application may be able to use to the system scheduler (cron/at on Posix, Windows Scheduler Service on Windows). While it should be possible for a PHP application to enqueue things into the webserver's user's crontab (as long as PHP isn't restricted to "safe mode"), I'm not sure that this is advisable. Most offline applications I know that need to schedule something spawn their own daemon to handle scheduled events, even if it sits idle most of the time.

I can't see why the frameworks shouldn't provide an API for scheduling tasks. This would have the advantage of being simple, integrated and portable, and it could negotiate to use the platform scheduler or fall back to spawning a daemon to dispatch events.

As mentioned before, Django's weakest link is its rudimentary template language. Content and data has to be digested and spoon-fed to the templates because the templating language doesn't provide the tools required to manipulate data arbitrarily. This doesn't allow the separation of presentation and controller code that is the purpose of a templating language. Instead, each template has a one-to-one dependence on the code which sets up the model. On the plus side, it's very easy to track where the thread of execution goes in these templates. A potentially dangerous issue is that the templating system defaults to not escaping data variables. This is effectively XSS unaware. Inverting the semantics such as to escape everything unless I tell you otherwise (to insert XML/SGML markup, say) would fix this.

The philosophy given by the Django documentation for the design of its templating is that

We wanted Django’s template language to be usable for more than just XML/HTML templates. At World Online, we use it for e-mails, JavaScript and CSV. You can use the template language for any text-based format.

This "one size fits all" approach is self-evidently misguided. XML and JSON need a guarantee of structural validity but they rarely need re-styling anyway. (X)HTML needs escaping. Javascript (other than JSON) should be static anyway. CSV barely needs an API, let along a templating language. Emails… well, OK, this is a fine templating language for text-only emails.

I'd also found my Django application hard to structure. Views (functions called by the Django context in response to different request URIs) shouldn't contain application code. Following that philosophy, my code is split mainly into four classes in two Django "applications", a handful of utility functions in the ORM models, and everything else is wrapped up in the 20 or so views. They don't do more than delegation and template context setup, but there be semantics in them there views, and those semantics aren't codified a form that can be nested and built into more advanced and interdependent web applications. For larger applications, Django will need code to modify and extend the framework, but there isn't a good API to plug in this code. It's a mixture of adding the class names to the config file, and giving modules special names within the package.

These issues tends to convey to me the general sense that Django code is hard to tie together. Overall, to build an application as elegantly structured as my current web shop code would require a sizable layer on top of the Django framework, with the occasional aspect of Django re-written too. But then, my current shop code has already borrowed some of the best aspects of programming with Django as well as a handful of other frameworks too

I have previously mentioned why I abhor PHP even as I developed Mauvespace in that language, but as I started this project PHP seemed utterly unconscionable. PHP doesn't offer anything apart from a dodgy syntax, incomplete object model and an API consisting of a basic CGI wrapper and bindings for a few library functions. Alternatives included my Python framework, but that is a long way from being ready for the mainstream. In fact it's been all but abandoned. I originally wrote it to serve static sites generated from XSL and XML, but then found myself bolting in little bits of CGI scripts and eventually writing a whole ORM.

I had also been looking at Zope, and particularly Zope 3, but Zope has a ridiculous learning curve. It does seem appropriate for more large-scale, extensible development though. It's just that most web application projects for SMEs don't need to be so enterprise that they require unit tests for everything and interfaces saying what each class will do as well as an implementation. I am looking into Zope but I don't expect to find a project that it will be suitable for very often.

So Django then. I'd seen Django before and discounted it because of its templating language, which was not what I wanted. I wanted to use XSL, obviously. XSL and XML can together be handed to browsers. This makes writing AJAX applications much easier: no special handlers are required to be able to pass XML data to the client. With the same requests, Javascript can create whatever REST model it likes.

But regardless, it looked more appropriate than anything else at this point.

So Django then. I was really impressed with Django initially. I say initially because while I managed to write about half of the application in the first two hours of using it, the rest has drawn out and out.

Django has an excellent ORM. However, it makes the mistake of trying to encapsulate all of SQL in the Python Object Model, which is a kind of antipattern in itself. This happens all the time with ORMs, because while it's easy to make database rows appear to be objects, SQL is designed to do clever things that you don't really want to re-implement in another layer just because you can abstract it. Django's ORM allows users to write code like Game.objects.get(id=the_game) to retrieve a single game. That's lovely. Then it allows you to write code like Game.objects.filter(category=this_category), and it lazily evaluates the query, even allowing you to chain these filters lazily. Which is fantastic.

Then you hit queries like finding objects with date ranges. In SQL this is easy.

SELECT * FROM games WHERE start <= NOW() AND end > NOW()

In Django this is munged into a horrific double-underscore variable name:

now=datetime.datetime.now()
Game.objects.filter(start__lte=now, end__gt=now)

If you're a programmer this should be fearing you with dread as to what comes next. Two sets or more of double-underscores starts performing joins, I believe.

So Django's ORM is a bit unwieldy, but this should come as no surprise because I think many ORMs are like this. One that I have seen that avoid this is Joomla's, where you just hand off an SQL query to the ORM and it just populates objects from the results. Trying to construct complex queries in an object-oriented way is likely to be harder and less maintainably than just writing the SQL query, and so I think this is the best solution in general.

Django's ORM libraries handily integrate with other bits of the system, such as administration screens, meaning almost no work has to be done to make your database rows entirely editable, with configurable permissions, from the admin screen. This is a big win for Django. There's no way you could even contemplate this with, say, Zope, because the ZODB stores arbitrary objects and an admin screen would have to deal with, basically, all of Python.

The annoyances I've found later on in the development of Django is that it's not intuitive. There's a vast wealth of features, and you have to keep looking at the documentation to find them all. There is a lot of magic involved. This is Python's fault, really. Magic is easy to do in Python, and a lot of developers won't have any issues with jumping in and creating magic for ever situation. This means that there is much less code to be written for performing many tasks, but magic has drawbacks. Developers have to understand what the magic is doing for them. In languages where magic is hard, developers can read the code and know, with very little context, exactly what it will do and why. With magic, the object model and even elements of the language can be warped. I'd agree that magic is good for an ORM: it should mean that you can get an object that magically persists. You don't need to know what's going on to make it persist. However, Django goes way too far with the magic. Admin screens are driven by magic variables. The URL configuration is slightly magical. I think permissions are almost entirely magical but I haven't got into that yet.

Then there's the templating language. I was set to hate this from the beginning. There is a much-vaunted feature called template inheritance (according to the documentation "the real power of Django templates". Apparently this allows one template to extend another. In practice this doesn't do anything more than allowing you to stuff some chunk of content into another template. The only situation this appears to be useful is in stuffing the main content of your page into a layout. Woo. It doesn't allow you to do really useful things, like calling templates that create generic page elements with arbitrary content. At least, without creating an inheritance tree of dozens of little template element files in a manner that is likely to be quite unmaintainable. Templates can read Python variables and attributes of Python objects that are passed to them. They can apply "filters" which are written in the core Python code. They can't do anything else, like even computing simple expressions. This is annoying; few other templating languages are so restricted.

In future it's likely that I'll write a Django middleware to do XSL transformation. Then I won't need Django's stupid templating. This doesn't sound like it will be too hard.

I'm currently trying to solve a problem with User authentication. I have a function, tickets_remaining() which I wanted to add to the User authentication class. But I can't do this. I need to create a separate Profile model and retrieve this instead.

So Django: on the whole, I'm impressed with it. The design is easy, extensible, and fast to get going with, but there are some caveats with the API and particularly the templating.

1. Get up to speed on RDF and get using it in applications. I am not a total stranger to RDF but I've not used it at all so far. The main focus of my effort for now is a new project called Mauvespace. Mauvespace is an open-source web application that is a cross between a semantic CMS for personal homepages and a full social networking service. I don't want to hype it too much now though until there is something to show. But I hope very soon to roll up all of my homepage stuff from Mauveweb into Mauvespace, then throw it open to other people to use it for the same thing, either on my server or on their own. This frees up the mauveweb.co.uk domain, which could become a place for web projects. Sorry about all the 'Mauve's. I guess I'm not very imaginative with names. Although, it works as a brand, I suppose.
2. Deploy some applications using Zope. My Python web applications are becoming increasingly Zope-like. The latest one I've been working on for a client is a self-contained web server, but that's partly because I wanted very careful handling of file uploads. I needed to remove file size and memory limits imposed by PHP, and I implement concurrent querying of the status of uploads, which allows me to provide AJAX progress bars. There are lots of parallels with Zope: that it's Python; that it's a web server; that any persistence is object-based (although in this application it's in-memory persistence; non-volatile data is retrieved from other network services mandated by the brief). Anyway, in 2007 I hope to transfer from ad-hoc Zope-like systems to Zope proper with all the advantages that brings. It's just a shame there have always been reasons not to so far. Unfortunately Mauvespace is PHP by necessity. PHP is the only language that enjoys widespread hosting support and I consider that vital.
3. Hack Inkscape. Inkscape is of course hugely important to my work and as a result I've become quite involved with making sure it meets my needs, mainly through bug reporting, feature requesting, and so on. I would like to stretch my C++ legs and improve things, if I find time. Incidentally Inkscape 0.45 has been bug hunted and is moving to feature freeze very soon. The headline news is the Gaussian blur feature but there are a plethora of other improvements too.
4. Continue the high standard of technical commentary on this blog Actually, I wish I could get it more organised and make it more accessible to people who aren't knowledgable web developers. But if it would be less personally useful to me if that was the case. So the status quo may have to suffice.

I do have a couple of ideas for web applications which I would consider writing/rewriting in Zope pretty soon.

I have only looked into Zope 2 so far, because that's what's available in Debian Stable, which is what is currently running on my web server. I am starting to understand some of the things Zope does, but I'm not at this point particularly sure why it does them.

The truly wonderful thing about Zope is the ZODB, Zope's Object DBMS. This was what really attracted me to Zope in the first place, working as I have been with Python CGIs with persistence mainly provided by my session class, which just pickles objects to temporary storage.

ZODB allows you to avoid dodgy object-relational mappers and just store real Python objects. This sounds wonderful, but in practice it's nowhere near this simple and if you have any experience of concurrent programming you will know why. Relational databases these days provide ACID for you quickly and effectively, and they can do this because the operations to perform are passed to the DBMS for it to execute. When operations must be performed in application code and there can't be a way to transparently persist the objects without running into concurrency issues. I couldn't understand how the ZODB was transparently solving these concurrency issues, so I read up a bit further on that subject, and as I understand it, it isn't. You get transactional locking, but the granularity is very coarse, so you need to do your own concurrency programming if you need performance. Happily it looks like there are some utility classes which implement some abstract datatypes in a concurrent way.

Zope provides 'products' which are object systems you can import into the ZODB hierarchy and combine to produce applications. To achieve anything useful I believe you must write your own products, which might manipulate other products (your own, built-in ones, or third-party) within the ZODB to achieve their task. For example, the folder object contains other objects and allows them to be accessed at a sub-URL.

Most of the examples involve DTML and ZPT, templating schemes which, although they could produce XML to plug into XSL if I wanted, just don't really fit the way I work.

There's also a Python Script object which might be useful as glue but is sandboxed quite severely.

At this point I must explain my main reservation with Zope 2. It's that the strategy of constructing object hierarchies by creating and configuring object instances in the ZMI (Zope's web-based administration interface) means that your web application is only stored within the ZODB. It depends on the contents of the database not only for data but code structure. Therefore none of your existing tools will work.

That means, for example, you can't use tools like Subversion to maintain a code repository or create a working copy. Perhaps it's possible to use it to maintain your Zope products, but not your entire web application. If you're already using Subversion, it's difficult for anyone to construct any kind of argument as to why you're better off not using it. ZODB's own versioning is not equivalent. In fact, almost anything you already have a tool to achieve, you must write a script to do in Python, or else do manually with the ZMI.

My other point of issue is that Zope's documentation describes in detail a principle called acquisition, which is really some magic with the Python object model to inherit functionality from other objects, a lot like subclassing, but not. The documentation has not, however, given me the slightest inkling as to what it's for.

I've just been reading a few bits and pieces about Zope 3 which suggest that it might be a more straightforward step, providing the bits of Zope that I do want as libraries without forcing me down a road of total Zopiness which would mean my abandoning of some of the tools that I have come to rely on.