Working on the web

Each month, David, Paul and I offer workshops for ‘Working on the web’, aimed at introducing staff to different aspects of Web 2.0 which might be useful in their research and teaching. Our original outline for these sessions can be seen over on the Learning Lab wiki.

A couple of things have reminded me recently that it might be useful to describe how I work on the web.

First of all, I use an up-to-date browser (Firefox or Chrome) with a few extensions. I block all advertising, using AdBlock, all trackers, using Ghostery and a password management extension, so I never use the same password on any two websites. Chrome allows me to synchronise all my preferences, bookmarks, passwords and other bits and pieces across different computers, so my experience on my desktop, laptop or home computer is the same. When using Firefox, I have the Sync extension installed, for the same reason.

Next, in terms of my basic set up, I have four useful ‘bookmarklets’: One for j.mp, which allows me to create a short URL for the current site, another for Readability, that makes reading long articles somewhat easier, one for delicious, to bookmark or ‘favourite’ sites, and a Posterous bookmarklet that allows me to quickly take clippings from web pages and post them to my Posterous site.

My Posterous site ‘things that stick’, is one of a few ways that I organise information on the web. I use Posterous almost exclusively for posting selected text (‘clippings’) from websites or PDF articles that make an impression on me. I use delicious for straightforward social bookmarking of a website, usually copying a piece of text from the site that best describes what it’s about. I use Google Reader to ‘Share’ whatever crops up in my feed reader that interest me. Whether I clip, bookmark or share, none of these actions is any kind of endorsement of the content but simply means the information is, in some way, of interest to me and I might want to come back to.

I share what is of interest to me by creating a ‘bundle’ from the RSS feeds of these three services in Google Reader. That bundle has a public web page and atom feed. However, all the items are presented in full text and therefore a hassle to get a quick overview of what’s been recently shared. So, I also aggregate the three sources to my own blog, ‘Elsewhere‘, where anyone can get a quick summary of the information I gather each day (and you can grab an RSS feed, too). I do this using the lifestream plugin for WordPress. This also means that through this process, the links I’m collecting ultimately come back to a site that I own and I have some kind of control over the retention of that data.

Google Reader is central to how I work on the web. I subscribe to news feeds from anywhere between 200 and 400 sites at any one time. Currently, it’s at a comfortable 230 subscriptions, which I read on my walk to and from work and occasionally during the day. I scan a couple of hundred headlines a day and click on about 10% of those headlines to read the article. This is my main method of reading the web.

I also use Google Reader to subscribe to every service I use on the web, so it’s a way of aggregating my own footprint on the web and keeping track of services I have used. The other reason for doing this is that Google Reader is searchable, so I can search over any of my activity on the web if I want to go back to something I read, create, shared or wrote.

Next, I have this work blog, which I use as a notebook more than anything else. I regularly refer back to it and search through it to remind me of the work I’ve done, ideas I’ve had and events I’ve been involved in. Whenever I have to report on my work, I refer back to this blog.

I use an Amazon ‘wishlist’ to maintain a list of books that look interesting and I might buy in the future. It’s a shame that there’s no RSS feed from wishlists. If there was, I’d add it to my daily bookmarks and clippings on my Elsewhere blog.

I use Mendeley to organise research papers in PDF format. Currently, I have over 500 PDF files synchronised across my work desktop and laptop (about 1.3GB). I moved to Mendeley, not for its social features, but simply because it renames and organises the files nicely on my hard drive and synchronises across computers. Before using it, I was in a mess.

I visit Wikipedia more than any other single website. It’s not perfect but its imperfections merely reflect our own imperfections and it is more perfect than any other collected source of information on the web.

I use Google docs for most of my non-blog writing these days. Funding applications, conference papers and articles I’m working on, all start off on Google docs and only move to Open Office if formatting requires.

I use slideshare to publish any presentations I give. I used to use Scribd until they starting charging people to download content from their site. When slideshare start charging, as I suspect they will, I’ll delete my account there, too.

On the subject of deleting accounts, I stopped using Twitter at the weekend. I’ve been trying to wean myself away from Twitter for months, having moved to using it largely for sharing links and as a news aggregator, picking up links from other people. I’ve never really liked it for conversation, finding the 140 character limit, well, limiting, in a demeaning sort of way. More recently, I’d created a private list of 20 or so people out of the 400 or so that I followed, who regularly pick up on sources of information I value, and this had become the extent of my experience using Twitter as I intentionally tried to wind down my use of it. Last weekend, I felt particularly overwhelmed with work and the intrusion that it can become at home, and so I deleted my account altogether. I know from past experience that not using it, rather than deleting it, wasn’t an option for me. I’d have simply ‘done a Stephen Fry’ and returned to it before too long, sorry addict that I’d become.

I’ve been on Twitter for a couple of years and had over 1000 followers, a few of whom are now real friends, though about half looked like people simply looking for re-follows, another large percentage were people who subscribed on mass to lists of people (usually EdTech lists) and quite a few more were people I’ve never had any contact with whatsoever. I’ve also found that my ability to concentrate has severely diminished over the last couple of years, with the constant distraction of having email/SMS/Twitter present in the back of my mind. Even turning off all notifications on my phone and computers hasn’t helped. Now I just use Google Reader to follow the RSS feeds of about 10 people on Twitter. It’s a bloody relief, to be honest. Here’s to being able to concentrate a little better from now on.

As with Twitter, I stopped using Facebook at the end of last year. The web is my social network and the above tools, my personal working learning environment.

Jailbreaking WordPress with Web hooks

As is often the case, I struggle at first glance to see the full implications of a new development in technology, which is why I so often rely on others to kick me up the arse before I get it. ((I am not ashamed to admit that I’m finding that my career is increasingly influenced by following the observations of Tony Hirst. Some people are so-called ‘thought-leaders’. I am not one of them and that is fine by me. I was talking to Richard Davis about this recently and, in mutual agreement, he quoted Mario Vargas Llosa, who wrote: “There are men whose only mission is to serve as intermediaries to others; one crosses them like bridges, and one goes further.” That’ll do me.))

Where I ramble about WordPress as a learning tool for the web…

I first read about web hooks while looking at WordPress, XMPP and FriendFeed’s SUP and then again when writing about PubSubHubbub. Since then, Dave Winer’s RSSCloud has come along, too, so there’s now plenty of healthy competition in the world of real time web and WordPress is, predictably, a mainstream testing ground for all of it. Before I go on to clarify my understanding of the implications of web hooks+WordPress, I should note that my main interest here is not web hooks nor specifically the real time web, which is interesting but realistically, not something I’m going to pursue with fervour. My main interest is that WordPress is an interesting and opportunistic technology platform for users, administrators and developers, alike. Whoever you are, if you want to understand how the web works and how innovations become mainstream, WordPress provides a decent space for exercising that interest. I find it increasingly irritating to explain WordPress in terms of ‘blogging’. I’ve very little interest in WordPress as a blog. I tend to treat WordPress as I did Linux, ten years ago. Learning about GNU/Linux is a fascinating, addictive and engaging way to learn about Operating Systems and the role of server technology in the world we live in. Similarly, I have found that learning about WordPress and, perhaps more significantly, the ecosystem of plugins and themes ((Note that themes are not necessarily a superficial makeover of a WordPress site. Like plugins, they have access to a rich and extensible set of functions.)) is instructive in learning about the technologies of the web. I encourage anyone with an interest, to sign up to a cheap shared host such as Dreamhost, and use their one-click WordPress offering to set up your playground for learning about the web. The cost of a domain name and self-hosting WordPress need not exceed $9 or £7/month. ((I am thinking of taking the idea of WordPress as a window on web technology further and am tentatively planning on designing such a course with online journalism lecturer, Bernie Russell. It would be a boot camp for professional journalists wanting (needing…?) to understand the web as a public space and we would start with and keep returning to WordPress as a mainstream expression of various web technologies and standards.))

… and back to web hooks

Within about 15 minutes of Tony tweeting about HookPress, I had watched the video, installed the plugin and sent a realtime tweet using web hooks from WordPress.

It’s pretty easy to get to grips with and if a repository of web hook scripts develops, even the non-programmers like me could make greater use of what web hooks offer.

Web hooks are user-defined callbacks over HTTP. They’re intended to, in a sense, “jailbreak” our web applications to become more extensible, customizable, and ultimately more useful. Conceptually, web applications only have a request-based “input” mechanism: web APIs. They lack an event-based output mechanism, and this is the role of web hooks. People talk about Unix pipes for the web, but they forget: pipes are based on standard input and standard output. Feeds are not a sufficient form of output for this, which is partly why Yahoo Pipes was not the game changer some people expected. Instead, we need adoption of a simple, real-time, event-driven mechanism, and web hooks seem to be the answer. Web hooks are bringing a new level of event-based programming to the web.

I think the use of the term ‘jailbreak’ is useful in understanding what HookPress brings to the WordPress ecosystem. WordPress is an application written in PHP and if you wish to develop a plugin or theme for WordPress you are required to use the PHP programming language. No bad thing but the HookPress plugin ‘jailbreaks’ the requirement to work with WordPress in PHP by turning WordPress’ hooks (‘actions’ and ‘filters’) into web hooks.

WordPress actions and filters, are basically inbuilt features that allow developers to ‘hook’ into WordPress with their plugins and themes. Here’s the official definition:

Hooks are provided by WordPress to allow your plugin to ‘hook into’ the rest of WordPress; that is, to call functions in your plugin at specific times, and thereby set your plugin in motion. There are two kinds of hooks:

  1. Actions: Actions are the hooks that the WordPress core launches at specific points during execution, or when specific events occur. Your plugin can specify that one or more of its PHP functions are executed at these points, using the Action API.
  2. Filters: Filters are the hooks that WordPress launches to modify text of various types before adding it to the database or sending it to the browser screen. Your plugin can specify that one or more of its PHP functions is executed to modify specific types of text at these times, using the Filter API.

So, if I understand all this correctly, what HookPress does is turn WordPress hooks into web hooks which post the output of the executed actions or filters to scripts written in other languages such as Python, Perl, Ruby and Javascript (they can be written in PHP, too) hosted elsewhere on the web.   In the example given in the HookPress video, the WordPress output of the action, ‘publish_post‘, along with two variables ‘post_title’ and ‘post_url’, was posted to a script hosted on scriptlets.org,  which performs the event of sending a tweet which includes the title and URL of the WordPress post that has just been published. All this happens as fast as the component parts of the web allows, i.e. in ‘real time’.

In other words, what is happening is that WordPress is posting data to a URL, where lies a script, which takes that data and creates an event which notifies another application. Because the scripts can be hosted elsewhere, on large cloud platforms such as Google’s AppEngine, the burden of processing events can be passed off to somewhere else. I see now, why web hooks are likened to Unix pipes, in that the “output of each process feeds directly as input to the next one” and so on. In the case of HookPress, the output of the ‘publish_post’ hook feeds directly as input to the scriptlet and the output of that feeds directly as input to the Twitter API which outputs to the twitter client.

Besides creating notifications from WordPress actions, the other thing that HookPress does (still with me on this ‘learning journey’ ??? I’ve been reading, writing and revising this blog post for hours now…), is extend the functionality of WordPress through the use of WordPress filters. Remember that filters in WordPress, modify text before sending it to the database and/or displaying it on your computer screen. The example in the video, shows the web hook simply reversing the text before it is rendered on the screen. ‘This is a test’ becomes ‘tset a si sihT’.

The output of the ‘the_content‘ filter has been posted to the web hook, which has reversed the order of the blog post content and returned it back to WordPress which renders the modified blog post.

Whereas the action web hooks are about providing event-driven notifications, the filter web hooks allow developers to extend the functionality of WordPress itself in PHP and other scripting languages.  In both cases, web hooks ‘jailbreak’ WordPress by turning it into a single process in a series of piped processes where web hooks create, modify and distribute data.

Finally, I’ll leave you with this presentation, which is all about web hooks.

In the presentation, there are two quotes which I found useful. One from Wikipedia which kind of summarises what HookPress is doing to WordPress:

“In computer programming, hooking is a technique used to alter or augment the behaviour of [a programme], often without having access to its source code.”

and another from Marc Prensky, which relates back to my point about using WordPress as a way to learn about web technologies in a broader sense. WordPress+HookPress is where programming for WordPress leaves the back room:

As programming becomes more important, it will leave the back room and become a key skill and attribute of our top intellectual and social classes, just as reading and writing did in the past.

PubSubHubbub: Realtime RSS and Atom Feeds

It’s made Dave Winer happy, which is no easy task, so I think PubSubHubbub is worth mentioning here. If it’s working as it should, this post should appear in my Google Reader, almost immediately after I’ve published it. That’s because PubSubHubbub is “a simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom [and RSS].” My blog feed is managed by FeedBurner which has already implemented the new protocol, as has Google Reader FriendFeed. They should therefore ‘talk’ to each other in realtime. Watch the video and you’ll see how it works. It’s pretty straightforward. It just takes a company the size of Google to push it through to adoption. The engineers say they were using it like Instant Messaging the night before the demo, which says something about how responsive this is. Technically, it should be another challenge to Twitter in that it allows for a distributed method of near realtime communication.  I’d like to see that. I feel like an idiot communicating within the confines of  Twitter, sometimes.