Archive for the ‘Frameworks’ Category

Satchmo

Monday, January 7th, 2008

Since I’ve been using the Django framework for web development I’ve been pointed in the direction of the Satchmo framework a few times. Satchmo is intended to be the framework for e-Commerce development built on top of Django. I’ve looked at Satchmo more than once trying to weigh up whether it would be a useful thing to use for projects, because an e-Commerce framework certainly should be useful. Satchmo doesn’t satisfy this expectation.

I have written an e-Commerce shop in Django. It took three days, much of which was spent writing the backend - basket, order processing and so on. I am absolutely certain that I couldn’t have done that with Satchmo. The first reason is that Satchmo isn’t orthogonal; it’s impossible to use Satchmo’s basket/ordering system without using Satchmo’s products front end. Front ends are very easy to write in Django and I needed something custom for this shop. Custom front ends are very hard to write in Satchmo - they involve writing custom extensions to the product model. The second reason is that Satchmo doesn’t use Enterprise Design Patterns like Money.

Satchmo also provides code that is very much outside its remit as a framework, like this strange configuration property which embeds Google Analytics. Google Analytics code is trivial to paste into the templates should I want to do so. Why would I need this built into the platform?

Satchmo doesn’t satisfy as a framework; it’s more of an off-the-shelf application regardless of what the authors claim. Now I have Django web shop code, I won’t be needing Satchmo.

Scheduling Events

Thursday, August 9th, 2007

There are several situations in web application programming where it is necessary to schedule events to happen in the future, outside of the request driven model. Some of the most common are these:

  • Expiring static files from the webserver. Some data can be cleaned up whenever a page is requested. On occasion, though, the application establishes the contract that a file will stay around for a fixed period of time. When access to these files is provided by the webserver (not through the application itself) then the files need to be deleted at a given future moment.
  • Time-based notifications. For example, if you deal with dates and times in your web application it’s sometimes necessary to actually notify users (most often, via email) at a given time. It’s clearly not acceptable to wait until someone hits a page (possibly hours or days later) to issue these notifications.
  • Syndication. Polling data on remote servers has to be done ready for when a user hits a page, because otherwise it can introduce an unacceptable delay while variously contactable remote hosts are queried.

In several of my web applications now I’ve come to a sticking point when it comes to scheduling events. As far as I know this is always left up to the developer to arrange. Scheduling events is considered outside their remit.

There are a few solutions I know of.

  • The application can provide a script which the administrator must schedule to be run periodically at install time. Drupal, for example, recommends adding a crontab entry which periodically wgets a script on the web site. In redistributable apps, many users will obliviously skip this step and wonder why the application won’t work.
  • Run scheduled tasks after serving each page. This approach doesn’t solve the above problems. In mod_php/perl/python applications this hogs a webserver thread too, which could degrade performance.
  • There are websites like webcron.org that will fetch a script on your server at intervals. It would be madness to rely on this in your own applications or suggest this as a solution for a redistributable applications, so it’s only suitable as a fallback if all else fails.
  • The application may be able to use to the system scheduler (cron/at on Posix, Windows Scheduler Service on Windows). While it should be possible for a PHP application to enqueue things into the webserver’s user’s crontab (as long as PHP isn’t restricted to “safe mode”), I’m not sure that this is advisable. Most offline applications I know that need to schedule something spawn their own daemon to handle scheduled events, even if it sits idle most of the time.

I can’t see why the frameworks shouldn’t provide an API for scheduling tasks. This would have the advantage of being simple, integrated and portable, and it could negotiate to use the platform scheduler or fall back to spawning a daemon to dispatch events.

More on Django

Monday, July 2nd, 2007

As I come to the end of my first project using Django, I can offer a slightly better picture of how Django actually measures up in the field. In general I found that the bulk of the application was easier than I’d expect, but the remainder was more time consuming than I’d expected. (more…)

Django

Friday, April 6th, 2007

For the past couple of weeks I’ve been developing a web application for a client in the Django framework. I looked at frameworks before I started, wondering what I could use to make the development of this project easier.

I have previously mentioned why I abhor PHP even as I developed Mauvespace in that language, but as I started this project PHP seemed utterly unconscionable. PHP doesn’t offer anything apart from a dodgy syntax, incomplete object model and an API consisting of a basic CGI wrapper and bindings for a few library functions. Alternatives included my Python framework, but that is a long way from being ready for the mainstream. In fact it’s been all but abandoned. I originally wrote it to serve static sites generated from XSL and XML, but then found myself bolting in little bits of CGI scripts and eventually writing a whole ORM.

I had also been looking at Zope, and particularly Zope 3, but Zope has a ridiculous learning curve. It does seem appropriate for more large-scale, extensible development though. It’s just that most web application projects for SMEs don’t need to be so enterprise that they require unit tests for everything and interfaces saying what each class will do as well as an implementation. I am looking into Zope but I don’t expect to find a project that it will be suitable for very often.

So Django then. I’d seen Django before and discounted it because of its templating language, which was not what I wanted. I wanted to use XSL, obviously. XSL and XML can together be handed to browsers. This makes writing AJAX applications much easier: no special handlers are required to be able to pass XML data to the client. With the same requests, Javascript can create whatever REST model it likes.

But regardless, it looked more appropriate than anything else at this point.

So Django then. I was really impressed with Django initially. I say initially because while I managed to write about half of the application in the first two hours of using it, the rest has drawn out and out.

Django has an excellent ORM. However, it makes the mistake of trying to encapsulate all of SQL in the Python Object Model, which is a kind of antipattern in itself. This happens all the time with ORMs, because while it’s easy to make database rows appear to be objects, SQL is designed to do clever things that you don’t really want to re-implement in another layer just because you can abstract it. Django’s ORM allows users to write code like Game.objects.get(id=the_game) to retrieve a single game. That’s lovely. Then it allows you to write code like Game.objects.filter(category=this_category), and it lazily evaluates the query, even allowing you to chain these filters lazily. Which is fantastic.

Then you hit queries like finding objects with date ranges. In SQL this is easy.

SELECT * FROM games WHERE start <= NOW() AND end > NOW()

In Django this is munged into a horrific double-underscore variable name:

now=datetime.datetime.now()
Game.objects.filter(start__lte=now, end__gt=now)

If you’re a programmer this should be fearing you with dread as to what comes next. Two sets or more of double-underscores starts performing joins, I believe.

So Django’s ORM is a bit unwieldy, but this should come as no surprise because I think many ORMs are like this. One that I have seen that avoid this is Joomla’s, where you just hand off an SQL query to the ORM and it just populates objects from the results. Trying to construct complex queries in an object-oriented way is likely to be harder and less maintainably than just writing the SQL query, and so I think this is the best solution in general.

Django’s ORM libraries handily integrate with other bits of the system, such as administration screens, meaning almost no work has to be done to make your database rows entirely editable, with configurable permissions, from the admin screen. This is a big win for Django. There’s no way you could even contemplate this with, say, Zope, because the ZODB stores arbitrary objects and an admin screen would have to deal with, basically, all of Python.

The annoyances I’ve found later on in the development of Django is that it’s not intuitive. There’s a vast wealth of features, and you have to keep looking at the documentation to find them all. There is a lot of magic involved. This is Python’s fault, really. Magic is easy to do in Python, and a lot of developers won’t have any issues with jumping in and creating magic for ever situation. This means that there is much less code to be written for performing many tasks, but magic has drawbacks. Developers have to understand what the magic is doing for them. In languages where magic is hard, developers can read the code and know, with very little context, exactly what it will do and why. With magic, the object model and even elements of the language can be warped. I’d agree that magic is good for an ORM: it should mean that you can get an object that magically persists. You don’t need to know what’s going on to make it persist. However, Django goes way too far with the magic. Admin screens are driven by magic variables. The URL configuration is slightly magical. I think permissions are almost entirely magical but I haven’t got into that yet.

Then there’s the templating language. I was set to hate this from the beginning. There is a much-vaunted feature called template inheritance (according to the documentation “the real power of Django templates”. Apparently this allows one template to extend another. In practice this doesn’t do anything more than allowing you to stuff some chunk of content into another template. The only situation this appears to be useful is in stuffing the main content of your page into a layout. Woo. It doesn’t allow you to do really useful things, like calling templates that create generic page elements with arbitrary content. At least, without creating an inheritance tree of dozens of little template element files in a manner that is likely to be quite unmaintainable. Templates can read Python variables and attributes of Python objects that are passed to them. They can apply “filters” which are written in the core Python code. They can’t do anything else, like even computing simple expressions. This is annoying; few other templating languages are so restricted.

In future it’s likely that I’ll write a Django middleware to do XSL transformation. Then I won’t need Django’s stupid templating. This doesn’t sound like it will be too hard.

I’m currently trying to solve a problem with User authentication. I have a function, tickets_remaining() which I wanted to add to the User authentication class. But I can’t do this. I need to create a separate Profile model and retrieve this instead.

So Django: on the whole, I’m impressed with it. The design is easy, extensible, and fast to get going with, but there are some caveats with the API and particularly the templating.

Plan for 2007

Monday, January 8th, 2007

New Year is a good time to look forward to the things we hope to achieve over the next year. So I thought I’d define now my main (technological) priorities for the year ahead so that I can get some sense of focus.

  1. Get up to speed on RDF and get using it in applications. I am not a total stranger to RDF but I’ve not used it at all so far. The main focus of my effort for now is a new project called Mauvespace. Mauvespace is an open-source web application that is a cross between a semantic CMS for personal homepages and a full social networking service. I don’t want to hype it too much now though until there is something to show. But I hope very soon to roll up all of my homepage stuff from Mauveweb into Mauvespace, then throw it open to other people to use it for the same thing, either on my server or on their own. This frees up the mauveweb.co.uk domain, which could become a place for web projects. Sorry about all the ‘Mauve’s. I guess I’m not very imaginative with names. Although, it works as a brand, I suppose.
  2. Deploy some applications using Zope. My Python web applications are becoming increasingly Zope-like. The latest one I’ve been working on for a client is a self-contained web server, but that’s partly because I wanted very careful handling of file uploads. I needed to remove file size and memory limits imposed by PHP, and I implement concurrent querying of the status of uploads, which allows me to provide AJAX progress bars. There are lots of parallels with Zope: that it’s Python; that it’s a web server; that any persistence is object-based (although in this application it’s in-memory persistence; non-volatile data is retrieved from other network services mandated by the brief). Anyway, in 2007 I hope to transfer from ad-hoc Zope-like systems to Zope proper with all the advantages that brings. It’s just a shame there have always been reasons not to so far. Unfortunately Mauvespace is PHP by necessity. PHP is the only language that enjoys widespread hosting support and I consider that vital.
  3. Hack Inkscape. Inkscape is of course hugely important to my work and as a result I’ve become quite involved with making sure it meets my needs, mainly through bug reporting, feature requesting, and so on. I would like to stretch my C++ legs and improve things, if I find time. Incidentally Inkscape 0.45 has been bug hunted and is moving to feature freeze very soon. The headline news is the Gaussian blur feature but there are a plethora of other improvements too.
  4. Continue the high standard of technical commentary on this blog :) Actually, I wish I could get it more organised and make it more accessible to people who aren’t knowledgable web developers. But if it would be less personally useful to me if that was the case. So the status quo may have to suffice.

Zope

Tuesday, November 21st, 2006

You might have noticed that the title of this blog includes the name Zope, but that I have, thus far, not so much as mentioned Zope. Not to worry, I have been looking into Zope, but it is a big subject, and it’s incredibly time consuming to just sit and read swathes of documentation to even get an inkling as to how to work with it.

I do have a couple of ideas for web applications which I would consider writing/rewriting in Zope pretty soon.

(more…)