From Accessibility to Zope

Was playing around with Inkscape last night and ended up doodling some clip art of some fruit and veg.

Enjoy!

You might have noticed that the title of this blog includes the name Zope, but that I have, thus far, not so much as mentioned Zope. Not to worry, I have been looking into Zope, but it is a big subject, and it’s incredibly time consuming to just sit and read swathes of documentation to even get an inkling as to how to work with it.

I do have a couple of ideas for web applications which I would consider writing/rewriting in Zope pretty soon.

(more…)

I installed Apache Batik for the first time last night. Batik is a Java library for SVG, particularly rasterising drawings. It did exactly what I needed (converting SVG graphs to PNG) without much fuss, either with installation or integration (I used the commandline rasteriser). I’d just not had a situation where I’d needed something other than Inkscape’s commandline before.

I did have to install some nicer ttf fonts as there weren’t any installed apart from Lucida in the JRE, but simply using aptitude to install an xfonts package worked.

It was a little slower than I had hoped, unfortunately, even for these very simple graphs, which means that working with this in web apps won’t be quite as straightforward. It was much faster doing batches than individual graphs.

Anyway, server-side rasterisation is one obstacle to SVG adoption which happily proves relatively simple to overcome.

The other day I doodled some ideas for themes for this blog. Not really based around web design. Just interesting concepts, hopefully.

My discussion about easy scripting of web apps is part of the subject of API design, which I do find very interesting. API choice is subjective, but is it that subjective? There should be examples of APIs everyone can point to and say “That is how/how not to do it”.

Obviously, I’m a fan of object-oriented APIs, but the ease of use of OO APIs differs greatly.

Python’s standard API is too flat. The modules are self-contained rather than having dependencies on one another. This means most functions will tend to return a primitive rather than a more suitable datatype. There is no consistency in naming, but that’s not too bad because of the interactive interpreter and help.

Java’s API is too nested. It is totally interdependent, but its power usually comes from composition of the objects rather than subclassing, so some of the packages get extremely complex. It is entirely based on design patterns and has a consistent naming policy which they only break occassionally (*cough* System.currentTimeMillis() *cough*). I think I prefer this to any other API I’ve used, despite areas of high complexity and still not having found a truly succinct way of using AWT and Swing (perhaps I will try a XUL implementation next time).

The worst API I’ve ever looked at is libxml2, minor faffing with which is required to get XSLT up and running in Python. It’s so bad it gives me a headache just looking at the API documentation! Attempting to do anything with XML without OOP is just a horrific thing to envisage and for this alone, libxml is contemptible.

I solved my problem mentioned in my last post by creating my own scriptable interface in Python.

Happily, I had decoded enough of the database format to be able to write a class - about 50 lines of Python - which encapsulates the configuration I needed to modify, and the ability to store it into the database (I didn’t fully implement reading the old configuration from the database though).

Then I was able to do the reconfiguration as simple object instantiation and member function calls. As expected, it was much easier than logging in as administrator for each CMS in turn, going through the interface to find the configuration page required, and updating and saving it.

There were a number of advantages:

• All the configuration to be input was collected at the bottom of the script, together on one page, so it was easy to check that everything was present and correct.
• The script can be re-run at any point, so it’s maintainable.
• The script performs some checks to ensure the new configuration is valid.

Obviously, it would be easier if Joomla offered this kind of capability so it didn’t have to be maintained separately in Python. Joomla is written in PHP, but that doesn’t mean you can just write a PHP script that loads up Joomla’s classes and tweaks them as I’ve described. PHP doesn’t have import semantics, so you need knowledge about what has to be included and in what order. It also doesn’t have a very strong object model so trying to make object manipulations correspond to data manipulations is inherently inconvenient. Finally, Joomla’s PHP objects are comprised, as far as I’ve examined, of monolithic functions that do a lot of procedural steps rather than small, reusable functions.

So I’m one step closer to defining exactly what I need here for administering web applications. I need to be able to write administration scripts quickly, without prior knowledge of the application’s internals (ideally using interactive introspection to obtain the knowledge I need).

1. An API which allows me to succinctly retrieve and manipulate the application’s data. This should be object-oriented so that for any object I’m handed, I know what operations I can perform with it. The structure of the object model should reflect the way the data is presented in the web app’s front end, so that I go blindly using what I see of the web app’s data model.
2. A very low overhead in terms of lines of code for getting up and running. I don’t want to learn code by rote just so I can bootstrap this API.
3. Implicit or succinct persistence for the objects I’ve retrieved. I can call store() on each object I modify, if I must, but implicit persistence (ie. everything is automatically updated when the script ends) will better allow the API to handle ACID for me.
4. No SQL queries. I don’t want to have to understand the database structure. I also don’t care about efficiency in this instance so just hand me a list of all the objects and I’ll filter it myself.
5. No using built-in types to represent abstract data. For example, no using associative arrays to represent objects. A class for everything, with useful member functions, please. I don’t want to have to work out the semantics or write any code that might already exist in the application.
6. A few CLI utilities that use this API. Not only can these perform useful tasks, but they prove the API works, is succinct, and gives examples of it in use that I can copy.

Note that the API doesn’t need to remain backwardly compatible. I’m happy to modify my management scripts. The aim is for the scripts to be as brief as possible, so it shouldn’t be hard to tweak them if the data model is changed (improved, we hope).

I was just working on a set of separate Joomla installations for a client today when I realised that I really needed to be able to run scripts against the different installations.

I was trying to install three different Mambots (one of Joomla’s three different types of extensions) in about 8 installations of Joomla - each with different database configurations and paths, and having started out with a Bash script to merely copy the plugin files into place, I realised that because automating the whole operation would involve reading a configuration file in PHP syntax and performing some queries in MySQL with it, coding this would probably take longer than installing the plugins manually.

There are not very many web apps which have any kind of scriptable API. In fact, I only really know of Mailman, which is only partly a web application. But it’s a feature I’ve used frequently in Mailman - there is a script bin/withlist which acquires locks and opens the list, allows you to modify the list as a Python object, and saves it on exit. Mailman provides a few CLI tools too which can be used in scripting but which are really only trivial examples of the power of the scriptable API.

When I began writing Mailhammer, my own announcement-only mailing list software, I took this scriptability even futher based on my positive experience with Mailman’s scriptable API. All of the working parts are implemented in Python, and the PHP is just an HTML wrapper which opens and talks to a CLI Python script over pipes. This means that the PHP is kept extremely simple, and the Python core is a very clean and simple API, and that the CLI can do everything reliably. It’s a cleanly divided implementation of an n-tier architecture. In fact in practice, I only use the web interface for viewing the data already in the database. Consequently, that interface isn’t very powerful - yet!

Python is well-suite for scriptable APIs - its interactive interpreter and neat object model mean that it’s easy to perform arbitrary operations interactively on complex, persistent data structures. In PHP web applications it might be more feasible to build an XML-RPC interface of some kind and provide a command-line client.

I don’t think that scriptability is considered as even a potential feature for almost any web application I’ve tried; their operation is tied inextricably to their unique interfaces.

For anybody developing a new web application please ask yourself this: will administrators using your software want to be locked in to your pretty and easy-to-use interface, or will they end up cursing you for failing to provide them with power beyond what HTML can provide?