From Accessibility to Zope

You can now dynamically reroute directions provided by Google maps by dragging the computed route!

This is unbelieveably cool stuff - it works like a desktop app, flawlessly and fast, and it’s powered by AJAX alone.

I have mentioned briefly work that I was doing to wrap file uploading in AJAX for a proper experience. Browser-based file uploads have been downtrodden over the past few years.

In client terms, file uploads work in almost exactly the same way as they have always done: the page blocks while the data is posted, and a very small progress bar shows up in the status bar. This is a user interface disaster for big files.

On the server side, the situation is more varied, but there is often little support for streaming of file uploads. In PHP, file uploads are read wholly into memory, parsed and saved out to a temporary folder before a script even gets called. The request must fit within both PHP’s file upload size limit and its memory limit. As far as I can tell, something similar happens in Zope although you can argue that Zope allows other standards for upload such as DAV and FTP natively. In plain CGI, of course, there is no handling of the uploads, so if you’re using a CGI wrapper, it can do whatever you want to handle this. Perl’s CGI.pm module allows a hook, at least. Python’s cgi module doesn’t, nor is it easy to subclass.

All in all, the situation of binding file uploads to form submissions, and processing of those in common server-side languages is wholly inadequate as file size gets large. File uploads are convenient because they are a commonly-supported fall-back, but the workarounds, although solving some of these problems, don’t have the simplicity of a browser-native solution.

In my recent project I looked at ways of working around these limitations. The best workaround for the client-side problems I have found so far is to perform the upload in an <iframe>, using AJAX queries to present a progress bar. This still has problems, notably that it’s one file at a time, both on the choosing and the uploading. In Firefox I can actually perform two concurrent uploads in different <iframes>, but the AJAX progress bar doesn’t then update.

Server-side, I wrote the whole thing as a webserver so that the AJAX queries could talk directly to the thread streaming the upload. Additionally I wrote my own parser to parse on-the-fly the data uploaded, so that the daemon knows what is uploading at any given stage. It works quite well, and the system is extensible in that it could combine a daemon that allows other forms of upload; feedback for these would also appear in the browser windows.

Even so, I wish that file uploading was something people were thinking about more. It’s central to so many web applications now.

There are numerous problems:

• File uploads are synchronous. Downloads can happen in the background in their own, but uploads can’t.
• File uploads don’t have a proper UI. Current browsers appear to show a tiny upload bar that isn’t really very accurate and doesn’t give data rates or estimated time remaining.
• Uploads are chosen one at a time.
• Javascript can’t be used for polish. The model that has empowered Web 2.0 improvements is that of taking an existing HTML/HTTP model and allowing it to be controlled by Javascript. However, there is no way into the uploading or the file selection processes with Javascript.

The most general solution I can see would provide a Javascript API for uploading. This would allow Javascript to show a (native) file chooser dialog, and instruct the browser on what to do with the files it returns. POST or PUT to the origin server seem useful, as does FTP upload. Clearly there are security concerns, but I fail to see how, as long as Javascript may instigate an operation, read upload statistics, but not read the filesystem, this presents a problem.

Perhaps an AJAX-style API could be along these lines:

//configure a native dialog to present to the user
var ufc=new UploadFileChooser();
ufc.setAcceptableFileTypes(['image/jpeg', 'image/png']);
var uploads=ufc.chooseFiles();

for each (var u in uploads)
{
u.onreadystatechange=doSomething; //callback
//this URL is constrained the origin server to prevent XSS
u.beginHttpPost('http://example.com/upload');
}

After this, the user could close the tab or leave the page, and the browser would upload the files in the background, perhaps with a progress bar appearing within the Downloads window. Note that it could queue the files rather than uploading them all at once, depending on user settings. The Javascript, and indeed the user, should be able to request that an upload is aborted. The Javascript should also be able to query the upload, using the object reference provided.

Just had an idea: how about using Javascript to record client-side usage of your website?

The principle is this:

1. Register Javascript listeners which construct a list of events, particularly mouse, scroll and click events, along with the time that the event was fired.
2. Register an unload event which posts the information as XML with AJAX to a script on the server when the user leaves the page.
3. Browsing sessions can be collated on the server using cookies.
4. Create a player, which reads the events as XML and renders them using a DHTML ‘cursor’ and/or by firing events within the DOM. Could have a time slider and fast-forward controls, etc, depending on how complex you want to get.

Voila - see exactly what people are doing with your site. I have knocked up a test which implements the first two steps, for mousemove events, and that much works, so the whole concept would be workable. I can imagine it would break down if your site uses plugins (or Javascript navigation, depending on how easy it is to replay the events accurately) but that’s a limitation you would have to live with.

There are obviously privacy concerns but this is relatively mild as no personal data would be recorded. Perhaps it could pop up a Javascript window.confirm() dialog asking if it’s OK to record your behaviour. But it would be a very useful tool for examining site usage, especially for commercial sites. This is the way modern marketing works. I leave it up to your conscience as to whether it’s ethical.

Well what do you know… what with all my recent work with client-side scripting and then today’s look at Mauvesoft Gallery and scanning for photos, I was just thinking about an AJAX-powered web-based Picasa clone when I stumbled across Google’s Picasa Web Albums, which I hadn’t seen before.

I’ve uploaded a few photos to it, and it’s OK I suppose, but it’s not as developed as some of the other Google webapps: it’s just a basic web gallery. It’s better than Gallery (Capital G) because Gallery is a mess these days. Gallery 2.0 is hugely bloated, and for all that bloat it isn’t very much more powerful and certainly not as easy to use as Picasa Web albums.

Still, I think Mauvesoft Gallery is a fairly good web gallery core, and it’s simple and hackable enough that I could probably rig up some AJAX over the top quite easily. It would certainly make for an easy-to-use administration interface, but I was thinking more of the on-the-fly searching, and Picasa’s wonderful ‘Timeline’ feature. I wonder whether the publishing to the world of photos is the main aim when people use web galleries, or is it that they just want access to all their photos wherever they are? (Or a third option: my brother, on holiday, uploads photos from Internet Cafes so that he can delete them from his camera to free up space.)

A fork of Mauvesoft Gallery is used to power the ImageChooser component in my e-Commerce software, and that is desperate for some attention. The UI is appalling at the moment. Perhaps I’ll download Dojo and see if that can speed up the development.

I’ve been working on this website recently which has really challenged my technical abilities.

I’ve put together a Javascript system, which I’m quite proud of, for implementing quite a lot of Flash-like effects with pure DHTML, with the aim of remaining accessible.

The complete system works like this:

1. You describe and lay out the pages however you want them with XHTML and CSS.
2. You describe transition timelines in an XML-based language. You can animate scalar CSS properties and colour values with cosine interpolation; you can assign attributes and CSS properties; and you can import and replace content in the document with AJAX. For accessibility, the idea must be to transition one page completely into another, although there can be some purely graphical effects in between.
3. The Javascript loads the transitions with AJAX and sets up event handlers to run the transition.

Easy! It’s like developing web pages in CSS, except that you get to animate your CSS properties. There’s also a bit of hackery to smooth over some IE issues.

I’ll post more about this when it’s a bit more polished.

That’s not the bit that has challenged my technical abilities. The technical challenge is working with the clients’ ideas. This job is actually under subcontract so there’s a chain of clients all wanting to get their way, but it seems to me that every time I implement something technically impressive, the clients see what I have achieved and assume that any small modification is an inherently small task. This is not universally true.

I think the site currently looks pretty naff. I’ve not had artisitic control on this one, and it’s a mess. The animations are distracting, and the clients keep asking for something ridiculous, like “make this image fill the screen”. That’s not something that can be achieved in an attractive way with a fluid layout. This was actually in the brief, which was why I originally started with a fixed layout, but one of their first demands was to move from a fixed to a fluid layout, and I can’t seem to get them to understand that’s the trade off.

This is the damage Flash has done to the world of web design. Clients want pointless effects and WYSIWYG modifications, because they’ve seen it on other sites, without knowing the actual situation with respect to accessibility or maintainability.

I’ve been experimenting with using Greasemonkey to rewrite the interface to a game I’ve been playing recently called Cities. I was trying this because I wanted to test how well a native SVG interface to a web application might work, and it was much easier to work with an existing web application - one that I use - than developing one as I go.

Here’s a screenshot of the standard Cities viewport (with CSS styles by Bebe):

And this the current status of my viewport implementation:

(more…)