From Accessibility to Zope

MVC as a pattern has many interpretations, but an interpretation I think is common goes as follows:

• Model - database abstraction/persistence
• View – templates
• Controller - all the code that interprets the request, interacts with models, and sets up and renders a template

There is a distinct problem with this, and it's that the controller is bloated with all manner of code  - things which don't seem bottom-up enough to be part of the model, or code to pre-chew model data for the benefit of dumb templates. This leads to controllers dozens of lines long that are difficult to read as a whole.

In my code (Django code, where controllers are called views) I have been try to avoid this, for a long while now. Since controllers constitute the glue between request, model interaction and setting up of the view, I read these frequently to determine how the application is glued together. Therefore I've found them to benefit from being as short and transparent as possible.

These facts should be immediately visible on reading the code of a controller:

1. What are the input parameters from the request (GET/POST/Cookies/Headers/Session Vars)
2. How those parameters are interpreted/validated (their domain)
3. What operation the request performs
4. What variables are passed to the template system

I think it's worth formularising this. My take would be something like this:

A controller should contain no code other than these distinct phases:

1. Unpacking the request parameters/validating the request.
2. Invoking an operation, defined elsewhere.
3. Setting up the context for template rendering.

These can even be labelled as such.

I've conflated unpacking/validating because these can generally be stated succinctly together, using trivial code. For example, in Django, you'd see code like this:

def category(request, category_id):
  category = get_object_or_404(Category, id=category_id)

which I think succinctly comprises these facts:

• The controller named category receives a parameter category_id
• The valid domain of category_id is the set of all Category ids
• If category_id is outside that domain, Http404 is raised

When identifying the business logic of a view, I've found that when the code enacting this grows to more than a couple of lines it's best either to be put into a form's .save() method (thus a form basically defines one operation on a mixed bag of unvalidated input), or into model methods (usually for simple model manipulation), or into a separate class that defines more complicated business logic.

When reading the template context setup, I want to know, when writing templates, what variables I have. Thus this should be explicit in the code of the controller I'm looking at.

I find this somewhat overly optimistic.

Today my Facebook account was hacked. Messages were sent to 42 of my friends, with a random subject and contents of the form:

hi! <recipient's first name>! <link>

All of the messages were shown as sent via Facebook Mobile, which, to my knowledge, I have never used.

I did several things:

1. I posted on my Facebook wall advising people not to open these messages.
2. I reported the intrusion to Facebook.
3. I changed my Facebook password.
4. Replied manually to every message sent warning people not to click on the links.

Below is the reply from Facebook. I've not replied yet, but it's frustrating that Facebook have not listened to a word I've said.

Subject: Re: Messages or Posts Were Sent From My Account, and I Didn't Send Them

Hi,

We have detected suspicious activity on your Facebook account and have reset your password as a security precaution.

Er… I told you about it. You're replying to an e-mail which I sent you about it. Detected my arse.

It is possible that malicious software was downloaded to your computer or that your password was stolen by a phishing website designed to look like Facebook. Please carefully follow the steps provided:

1. Run Anti-Virus Software: If your computer has been infected with a virus or with malware, you will need to run anti-virus software to remove these harmful programs and keep your information secure.

For Microsoft
http://www.microsoft.com/protect/viruses/xp/av.mspx
http://www.microsoft.com/protect/computer/viruses/default.mspx

For Apple
http://support.apple.com/kb/HT1222

As I told you in my e-mail, I run Linux and it is up-to-date.

2. Reset Password: From the Account Settings page, you will need to create a new password. Be sure that you use a complex string of numbers, letters, and punctuation marks that is at least six characters in length. It should also be different from other passwords you use elsewhere on the internet. Here is your new login information:

<redacted>

As I told you in my e-mail, I have already changed my password. Changing it again and sending it to me in cleartext e-mail is actually making the security of my account worse.

3. Secure Email: Make sure that any email addresses associated with your account are secure, since anyone who can read your email can probably also access your Facebook account. If you believe someone has accessed one of your email accounts, you should change its password.

As I told you in my e-mail, I don't believe anyone has access to my e-mail.

4. Never Click Suspicious Links: It is possible that your friends could unknowingly send spam, viruses, or malware through Facebook if their accounts are infected. Do not click this material and do not run any .exe files on your computer without knowing what they are. Also, be sure to use the most current version of your browser as they contain important security warnings and protection features.

As I said in my e-mail, my operating system is Linux and it is up-to-date. I cannot run any .exe files without serious difficult. In practical terms, it is very unlikely to have been compromised.

5. Log in at Facebook.com: Make sure that when you access the site, you always log in from a legitimate Facebook page with the facebook.com domain. If something looks or feels suspicious, go directly to www.facebook.com to log in.

Please. If I want to visit Facebook I select it from the AwesomeBar. I don't even receive e-mails from Facebook any more because I've disabled them, so I'd spot a phishing attack a mile off.

6. Learn More: Please visit the following page for further information about Facebook security and information on reporting material http://www.facebook.com/security

Wow, practical.

Finally, if this did not resolve your issue, please revisit the Help Center to select the appropriate contact form and submit a new inquiry:

http://www.facebook.com/help/?ref=pf

So that you can ignore what I say all over again?

Thanks,

The Facebook Team

Thanks for nothing.

These e-mails include random links, and it's probably that the nature of the attack could be uncovered by finding out more about what these links contain. It seems very probable that the page you would see will try in some way to continue the attack. That is the definition of a worm: an attack that propagates itself over the network. I tried downloading the contents of a link with wget. It timed out.

Worms are not unknown on Facebook. As always, think very carefully before clicking on untrusted links, installing untrusted apps, and check carefully that the site you are entering your credentials into is the one you expect.

My thanks to Sammy and Marit for alerting me to the attack.

Amazon have made a bit of an mess of building their thumbnails. On their homepage I was greeted with these:

The moiré pattern of blurriness is an artifact – evidence of the fact that these "Look inside" thumbnails are caused by resizing already thumbnailed images – probably the thumbnail of the book cover without the "Look inside" banner. To avoid this on your sites, you need to build thumbnails from a sufficiently high-resolution image – ideally a high-resolution original. In practice, it can be faster and less memory-hungry to thumbnail from a medium-sized image, and this will generally not show visible artifacts. Of course, if you've already got a high-resolution image loaded into memory, you can side-step all of the quality issues by building all of the thumbnails you might need from it at once. Note also that you need to resize down enough to hide any JPEG compression artifacts.

To understand how the tell-tale moiré pattern comes about, let's imagine the source and destination pixel grids:

When we overlay them you can see the moiré pattern appearing.

Where the grid intersections are aligned, one source pixel maps fairly closely to a destination pixel, which makes that spot in the thumbnail crisp. But as you move away from those spots and the error builds up, the grid intersections disalign, and one source pixel is smeared over four destination pixels. That makes for a blurry spot.

Thumbnail screenshots of websites seem to improve web usability enormously. For me, seeing a thumbnail triggers clearer and faster recognition than a domain or a name alone. Favicons also help when I've used the site enough to become accustomed to it. The GooglePreview Firefox extension is a favourite of mine for this reason.

There are now quite a number of websites which allow free website thumbnails. While these services are pretty good, and I recommend using them, these services require a huge amount of bandwidth to load the websites and serve the thumbnails, a lot of CPU time to render the websites, and a lot of storage to store them all. This means they consume money and the companies running them place a variety of restrictions on what can be done with the thumbnails. Also you very frequently find thumbnails don't yet exist or no longer exist, and the thumbnail service serves up some advertising instead, which is bad for usability. Perversely, it's for the same infrequently visited sites that it's hardest to remember that thumbnails get purged quickest.

If Google or another large search engine entered this market they could make a fast and free service that would be self-supporting. They are the only people who are making vast amounts of money enhancing the web – because a better web drives more business through their main search engine.

However, in the absence of that, I wonder if we shouldn't turn to distributed technologies to make the business of understanding where a link takes you an innate part of web standards, rather than a bolt-on service controlled by a vendor.

You could imagine a web standard similar to the favicons system, where thumbnails of the website are available at standard sizes – say 128×128 or 256×256 – at /thumbnail128 and /thumbnail256, but this places the onus on the publisher to create the screenshots and keep them up to date. Even worse, it's not a great idea to trust the website themselves. Shock sites, porn sites or scam sites could benefit from misleading users into visiting a site.

One solution might be a distributed network for website thumbnails. A lot of research and development has been done in the area of DHTs particularly to improve the performance and decentralisation of peer-to-peer networks. A client could look up a URL in a DHT to obtain a URL for a thumbnail of that website.

There is also a way of generating thumbnails in a distributed manner: web browsers. There are so many web browsers visiting so many websites that if you could tap into only a tiny fraction of them – with, say, a Firefox extension that generates and uploads thumbnails using <canvas> (assuming you can work around the privacy implications) – you could get good coverage quickly. Because it piggy-backs onto the normal web-browsing experience, it uses very little extra bandwidth than what users were already using.

Apparently, improved developer control of browser-native file uploads, something I wishlisted back in 2007, is going to be available in the upcoming Firefox 3.5.

My company, Mauve Internet, is looking to take on staff in the Chichester/Portsmouth area. I am looking for a Django developer, so if you have experience with that, or perhaps only a little Python experience but are enthusiastic to learn more, please check out the job details or perhaps get in touch.

Mauve Internet is currently based in Chichester but I've been looking at offices in Portsmouth and Havant too, perhaps I'm even leaning that way. Also, the job spec I've written up is fairly narrow. In truth, we may have positions open for a much wider range of talents and levels of experience in the field of web design and web development. I'm be happy to consider all applications accompanied by a CV.

I've overhauled Mauvesoft, my programming projects website. Check it out.

Programming a calendar sounds deceptively easy. And it is, until you come to realise that there's very little point in displaying a calendar that doesn't show information about events and periods. You have a potentially overlapping set of periods to display, each spanning days or months. It becomes much more complicated.

At the moment I'm programming a calendar for the booking of accommodation, which is particularly complicated because a) you book nights, not days, and month planners have cells for days, not nights, and b) the dates that are available are the dates not booked, not the dates booked.

I'm using a simpler approach, converting all calendar periods into a stream of events in date order. The interface between producers and consumers of calendar events looks like this:

class CalendarListener(object):
  def start_month(self, month):
    """Called before the first day of the month, and before any periods in that month."""
   
  def end_month(self, month):
    """Called after the last day of the month, and after any periods in that month."""
   
  def start_day(self, date):
    """Called once for each day to display"""
   
  def start_period(self, date, period):
    """Called before the day in which the period begins"""
   
  def end_period(self, date, period):
    """Ends the previously started period"""

This interface makes it very easy to produce, filter, and consume calendar data. What was previously a complicated process of intersecting, splitting, joining, structuring and outputting date ranges suddenly becomes very simple. All of the events received via this interface are guaranteed to be in chronological order, so no date comparison is needed. Almost all calendar operations can be performed with a simple state machine.

A consumer that renders to HTML, for example, is as simple as this:

class MonthRenderer(CalendarListener):
  def __init__(self):
    self.buf = StringIO()
   
  def start_month(self, month):
    print >>self.buf, """<div class="month"><h4>%s</h4>
      <img class="week" src="/assets/cal/week.png" alt=""/>""" % month.name()
   
    w = month.first_day().weekday()
    if w:
      print >>self.buf, '<div class="padding" style="width: %dpx"></div>' % (w * 21)
 
  def end_month(self, month):
    print >>self.buf, "</div>"
   
  def start_day(self, date):
    print >>self.buf, '<span class="day">%d</span>' % date.day

(Note: date and datetime are standard Python classes. Month, however, is my own class. Also, some people use a table rather than CSS for this; that's obviously a fairly simple alteration.)

It took me quite a few false starts before I realised the relative simplicity and convenience of this pattern, which is why I wanted to recommend this. It's very easy to fall into a trap of building complexity and tackling problems using ever-more complicated calendar classes and processors and never take the step back to find a better approach.

The naïve approach for programming a calendar is to write a function, say, print_month() which renders a month of a calendar. Then call this 12 times. Then wrap it up in a class so you can subclass it to retrieve a list of events and modify output. This quickly became excessively complicated, as I wrote methods to chop and join periods together, work out what the formatting of each day should be, and render it.

Alas, the calendar also requires Javascript, and doesn't benefit quite as much from an event-driven approach because it needs to operate on the structured HTML DOM.

The current generation of social networks are based on an assumption that giving you reams of data about what other people are doing and have done really puts you in touch with other people. As a user, I do get the impression that I am in touch with people, even though I may not actually be communicating with them. So we don't always bother to make stuff happens.

There are social networks that tell you where friends have been, some that tell you where they are, some that tell you what they are doing right now and some that tell you where they will be. But very few, it seems, that tell you where to be.

Taking Facebook as an example (as it's the only social network I'm using heavily at the moment), it does not make it easy to set up things to do. Creating events is a very laboured process. It takes perhaps 15 minutes to set up an event. It's an individual rather than a collaborative task. Invitations to events get ignored because of the way they are delivered. People get blanketed with invitations that they don't want. And I can't even set up an event until I've set up a group to arrange the event.

Social calendaring is not a new concept as the promise of electronic calendars has always included ease of scheduling, via e-mail invitations of some sort. Google Calendar and 30 Boxes represent the state of the art in this regard, which is simply calendar sharing and event invitations.

The thing social calendars really should address is scheduling of events, because I'm lazy and also busy and I always say things like "we really should …" but it never happens.

My ideal social calendar could fulfil these user stories:

• Find me something to join in with on any given day.
• Schedule things with the knowledge of when I'm most likely to be free or busy, even when nothing is scheduled.
• Arrange online games with my brother in Australia (7 hours ahead in summer or 9 hours in the winter).
• Pick a date and time to do a thing I want to do, with friends who want to and who may have to travel to do it.
• Remind me to book the venue for an event.
• Book out an event I'm hosting.
• Nail down the fuzziness inherent in saying something like "Let's have dinner on Thursday evening" so that we can say "Dinner at 8pm, and Alice will be joining us at around 10pm for drinks".
• Suggest when to actually go to bed so that I can get up next morning.
• Pin-point exactly where an event is so that I can work out how to get there.
• Don't keep trying to schedule things my skint friends can't afford to do.
• Suggest things I might like to do.
• Mildly favour a schedule where I can watch my favourite TV programmes.
• Create entirely new groups of local people with similar tastes (say Buffy, or Linux) in such a way as to be actually kind of fun and neither awkward or annoying.

As is perhaps evident, I'm a strong believer in heuristic tools that do really innovative stuff. What are the chances that all of the above are possible in a calendar application that doesn't automatically book me to go pole-dancing in Alaska moments before it has me watching Mork and Mindy with total strangers in my home on a Friday night? Hmm.