Category: Web

Jailbreaking WordPress with Web hooks

Joss Winn / / Fun, Open Source, Software, Standards & Specs, Twitter, Web

As is often the case, I struggle at first glance to see the full implications of a new development in technology, which is why I so often rely on others to kick me up the arse before I get it. ((I am not ashamed to admit that I’m finding that my career is increasingly influenced by following the observations of Tony Hirst. Some people are so-called ‘thought-leaders’. I am not one of them and that is fine by me. I was talking to Richard Davis about this recently and, in mutual agreement, he quoted Mario Vargas Llosa, who wrote: “There are men whose only mission is to serve as intermediaries to others; one crosses them like bridges, and one goes further.” That’ll do me.))

Where I ramble about WordPress as a learning tool for the web…

I first read about web hooks while looking at WordPress, XMPP and FriendFeed’s SUP and then again when writing about PubSubHubbub. Since then, Dave Winer’s RSSCloud has come along, too, so there’s now plenty of healthy competition in the world of real time web and WordPress is, predictably, a mainstream testing ground for all of it. Before I go on to clarify my understanding of the implications of web hooks+WordPress, I should note that my main interest here is not web hooks nor specifically the real time web, which is interesting but realistically, not something I’m going to pursue with fervour. My main interest is that WordPress is an interesting and opportunistic technology platform for users, administrators and developers, alike. Whoever you are, if you want to understand how the web works and how innovations become mainstream, WordPress provides a decent space for exercising that interest. I find it increasingly irritating to explain WordPress in terms of ‘blogging’. I’ve very little interest in WordPress as a blog. I tend to treat WordPress as I did Linux, ten years ago. Learning about GNU/Linux is a fascinating, addictive and engaging way to learn about Operating Systems and the role of server technology in the world we live in. Similarly, I have found that learning about WordPress and, perhaps more significantly, the ecosystem of plugins and themes ((Note that themes are not necessarily a superficial makeover of a WordPress site. Like plugins, they have access to a rich and extensible set of functions.)) is instructive in learning about the technologies of the web. I encourage anyone with an interest, to sign up to a cheap shared host such as Dreamhost, and use their one-click WordPress offering to set up your playground for learning about the web. The cost of a domain name and self-hosting WordPress need not exceed $9 or £7/month. ((I am thinking of taking the idea of WordPress as a window on web technology further and am tentatively planning on designing such a course with online journalism lecturer, Bernie Russell. It would be a boot camp for professional journalists wanting (needing…?) to understand the web as a public space and we would start with and keep returning to WordPress as a mainstream expression of various web technologies and standards.))

… and back to web hooks

Within about 15 minutes of Tony tweeting about HookPress, I had watched the video, installed the plugin and sent a realtime tweet using web hooks from WordPress.

https://videopress.com/v/25KHD2dF

It’s pretty easy to get to grips with and if a repository of web hook scripts develops, even the non-programmers like me could make greater use of what web hooks offer.

Web hooks are user-defined callbacks over HTTP. They’re intended to, in a sense, “jailbreak” our web applications to become more extensible, customizable, and ultimately more useful. Conceptually, web applications only have a request-based “input” mechanism: web APIs. They lack an event-based output mechanism, and this is the role of web hooks. People talk about Unix pipes for the web, but they forget: pipes are based on standard input and standard output. Feeds are not a sufficient form of output for this, which is partly why Yahoo Pipes was not the game changer some people expected. Instead, we need adoption of a simple, real-time, event-driven mechanism, and web hooks seem to be the answer. Web hooks are bringing a new level of event-based programming to the web.

I think the use of the term ‘jailbreak’ is useful in understanding what HookPress brings to the WordPress ecosystem. WordPress is an application written in PHP and if you wish to develop a plugin or theme for WordPress you are required to use the PHP programming language. No bad thing but the HookPress plugin ‘jailbreaks’ the requirement to work with WordPress in PHP by turning WordPress’ hooks (‘actions’ and ‘filters’) into web hooks.

WordPress actions and filters, are basically inbuilt features that allow developers to ‘hook’ into WordPress with their plugins and themes. Here’s the official definition:

Hooks are provided by WordPress to allow your plugin to ‘hook into’ the rest of WordPress; that is, to call functions in your plugin at specific times, and thereby set your plugin in motion. There are two kinds of hooks:

  1. Actions: Actions are the hooks that the WordPress core launches at specific points during execution, or when specific events occur. Your plugin can specify that one or more of its PHP functions are executed at these points, using the Action API.
  2. Filters: Filters are the hooks that WordPress launches to modify text of various types before adding it to the database or sending it to the browser screen. Your plugin can specify that one or more of its PHP functions is executed to modify specific types of text at these times, using the Filter API.

So, if I understand all this correctly, what HookPress does is turn WordPress hooks into web hooks which post the output of the executed actions or filters to scripts written in other languages such as Python, Perl, Ruby and Javascript (they can be written in PHP, too) hosted elsewhere on the web.   In the example given in the HookPress video, the WordPress output of the action, ‘publish_post‘, along with two variables ‘post_title’ and ‘post_url’, was posted to a script hosted on scriptlets.org,  which performs the event of sending a tweet which includes the title and URL of the WordPress post that has just been published. All this happens as fast as the component parts of the web allows, i.e. in ‘real time’.

In other words, what is happening is that WordPress is posting data to a URL, where lies a script, which takes that data and creates an event which notifies another application. Because the scripts can be hosted elsewhere, on large cloud platforms such as Google’s AppEngine, the burden of processing events can be passed off to somewhere else. I see now, why web hooks are likened to Unix pipes, in that the “output of each process feeds directly as input to the next one” and so on. In the case of HookPress, the output of the ‘publish_post’ hook feeds directly as input to the scriptlet and the output of that feeds directly as input to the Twitter API which outputs to the twitter client.

Besides creating notifications from WordPress actions, the other thing that HookPress does (still with me on this ‘learning journey’ ??? I’ve been reading, writing and revising this blog post for hours now…), is extend the functionality of WordPress through the use of WordPress filters. Remember that filters in WordPress, modify text before sending it to the database and/or displaying it on your computer screen. The example in the video, shows the web hook simply reversing the text before it is rendered on the screen. ‘This is a test’ becomes ‘tset a si sihT’.

The output of the ‘the_content‘ filter has been posted to the web hook, which has reversed the order of the blog post content and returned it back to WordPress which renders the modified blog post.

Whereas the action web hooks are about providing event-driven notifications, the filter web hooks allow developers to extend the functionality of WordPress itself in PHP and other scripting languages.  In both cases, web hooks ‘jailbreak’ WordPress by turning it into a single process in a series of piped processes where web hooks create, modify and distribute data.

Finally, I’ll leave you with this presentation, which is all about web hooks.

Using Web Hooks

View more presentations from Jeff Lindsay.

In the presentation, there are two quotes which I found useful. One from Wikipedia which kind of summarises what HookPress is doing to WordPress:

“In computer programming, hooking is a technique used to alter or augment the behaviour of [a programme], often without having access to its source code.”

and another from Marc Prensky, which relates back to my point about using WordPress as a way to learn about web technologies in a broader sense. WordPress+HookPress is where programming for WordPress leaves the back room:

As programming becomes more important, it will leave the back room and become a key skill and attribute of our top intellectual and social classes, just as reading and writing did in the past.

Open Education: Talis Incubator Proposal

Joss Winn / / Commons, Data, Fun, Funding, Libraries, Mashups, Open Education, Projects, Web

Back in May, I woke up with an idea in my head which, in a slightly modified form, I’d now like to try and find funding for. ((I figure that if I repeat this idea enough times, someone will see that it’s worth funding ;-))) The idea is based on work we’re doing on our JISCPress project, which itself is based on work Tony and I have been doing with WriteToReply since February. In my original blog post, I proposed that WordPress Multi User ((and here I’ll repeat what is becoming my mantra: ‘the same software that runs six million blogs on wordpress.com’ )) and Scriblio, a set of plugins for WordPress which allows you to import an OPAC library catalogue and benefit from all the advantages of the WordPress ecosystem, would together allow libraries to host independently branded catalogues on an open, union platform.

Imagine that JISC, Talis or Eduserv offered such a platform to UK university libraries. It could be a service, not unlike wordpress.com, where authorised institutions, could self-register for a site and easily import their OPAC, apply a theme, tweak some CSS, choose from a few useful plugins, and within less than a day or two, have a branded, cutting-edge search and browse interface to their OPAC, running under their own domain.

Paul and I gave a Lightening Talk about this at Mashoop North, which I present to you below.

Bibliografreedom

View more presentations from Joss Winn.

Slide four is the useful one. It show the various slices of the platform and, by implication, the various uses each layer offers.  The bottom slice shows the OPACs converge with WPMU to the benefit of the institution. It’s a nice, easy, hosted service that would offer an end-user experience not unlike the one that Plymouth State offer to their users. The middle slice – the WPMU bit – is where the OPACs converge together in union, under a single administrative interface that is easy to manage, widely used and supported. For $5000/year, Automattic, the company that leads the development of WordPress and runs wordpress.com, would provide support and advice with a six hour SLA. On top of that, anyone with a knowledge of PHP, can quickly learn the guts of WordPress, as Alex who’s working on JISCPress, will testify. My point is that this is a well tested and widely understood technology.

Now, once you have one or more OPACs hosted on WPMU, you bring together a lot of library catalogue data into one database and the platform’s web analytics (i.e. usage trends) can be a rich source of data for learning about what library users are looking for. Each library, would have access to their own analytics, while the analytics for the entire platform would also be collected. I do this on our university WPMU installation.

The next slice in our diagram, shows a few different ways of getting data out of the platform (and this would also apply to each individual catalogue site, too).  First, you can see that the platform as a whole could act as a union catalogue where, from a single site, users could search across library holdings. That union catalogue would have all the useful features of WordPress, too. Next to that, you can see Triplify, a nice little web application that transforms a relational database into RDF/N3, JSON and Linked Data. Triplify could re-present the data in each catalogue as semantic data and this could be subsequently hosted on the Talis platform.  We’re already doing this with JISCPress. Every night, changes to any of the library catalogue data could be pushed to Talis, where the data can be queried and mashed up using the Talis API. Finally, don’t forget good old RSS and Atom feeds, which are available for almost every WordPress endpoint URL, as I’ve previously documented.

Given the work we’ve done on JISCPress, which covers our experience with WPMU and Triplify, I think that a demonstrator prototype, using entirely open source software, could be developed within the constraints of the Talis Incubator fund. I canvassed my original idea to the Scriblio mailing list and had positive and useful feedback from Ross Singer at Talis. Leigh Dodds at Talis also sees potential in the use of WPMU and Triplify, although I understand that neither of these people are endorsing the idea for the Talis Incubator fund, but their interest has been encouraging.

So, what I’m proposing is that Paul and I work with Casey Bisson, the Scriblio developer, on a short project to get this all up and running. In my mind, Scriblio needs some more work to make the set up process easier for a variety of library catalogues and the last time I looked, it needed documenting better, too. I think that the maximum of £15,000 from the Talis fund is workable. In fact, I’d like to bring it down a little to make it more attractive to the judges. Paul would bring his knowledge and expertise from working with our university library catalogue, I would bring what we’ve learned from JISCPress and could manage the WPMU server side of things and the project in general, as well as write documentation, while Casey could be funded to spend some dedicated time fine tuning Scriblio to meet our objectives.

So what do you think? A wordpress.com like platform for library OPACs that pushes semantic data to the Talis platform. Each catalogue remains under the control of its owner institution, while contributing to a wider union OPAC that will benefit users and offer the library community some useful analytics. The platform as a technology, would be as flexible as WordPress itself is, so additional features could be developed for the platform by other future projects. Only last week, Tony was discussing on his new Arcadia project blog, how it would be useful to be able to capture library catalogue links as QR codes. Well, using WordPress in the way I’ve described, we could implement that across every UK HEI Library catalogue in a snap using this plugin. Hoorah!

There is a tension between being relevant and being reputable

Joss Winn / / Google Search, Web

…or ‘how to get read on the World-Wide-Web.’

This is a presentation about Search Engine Optimisation (SEO), but it is also about literacy and reputation in the age of the Internet. It is about how to understand and write well for the web so that like-minded people can learn about what you’ve got to say and be compelled to tell others about what you’ve got to say, too.

Although it’s not aimed at scholarly writing, that doesn’t matter. To Google’s crawlers, HTML source code is HTML source code, whether you publish articles about research into HIV or have something pointless to say about the latest gadget. No matter what the content is about there are literary as well as technical observations that can improve your communication and the impact of your writing.

Much of the presentation elaborates on this: “There is a tension between relevance and reputable.” It’s interesting.

Ten reasons why you should pay attention to the geeks because actually they have something quite important to say which us non-geeky people should be listening to

Joss Winn / / Data, Mashups, Web

Re-broadcasting Mike Ellis’ recent presentation…

Don't Think Websites, think data

View more documents from Mike Ellis.

Commons based peer-production: One minute of Wikipedia edits

Joss Winn / / Commons, Data, Web

The technical conditions of communication and information processing are enabling the emergence of new social and economic practices of information and knowledge production. ((The Wealth of Networks: Direct link))

You may have read Yochai Benkler’s book, The Wealth of Networks, where he discusses Wikipedia as an example of commons-based peer-production. Did you know that you can see this relatively new model of knowledge and economic production live, in real-time? The video below is just one minute of Wikipedia edits recorded from the live changes on the irc.wikimedia.org #en.wikipedia channel. Using the IRC channel, you can watch Wikipedia being created as it happens, which means you can see the incremental production of collective knowledge as it happens. I recommend full-screen HD to see the detail as it passes up your screen. There are different channels for the different language versions. I chose the English version.

The Wikimedia site provides detailed statistics about the use of their sites, although the English Wikipedia statistics stop at October 2006 🙁 Perhaps there’s just too much activity on that site for them to collect and measure?

A lot of people still have an aversion to Wikipedia, but I don’t think they get it. Wikipedia is completely open to anyone to contribute. If you don’t think it’s good enough, ((See the famous Nature article which compared Wikipedia to Encyclopedia Britannica [PDF])) isn’t it your (moral?) responsibility to correct and improve it? Like it or not, as a single source, it has by far the widest reach of any web-based learning resource and although I don’t have the time to substantiate this, I bet that after Google, it’s the second online resource that students visit when beginning their research. ((Via Twitter, AJCann just pointed me to some research he’d done which shows that 100% of his student cohort use Wikipedia)) If you challenge what’s happening on Wikipedia, you’re fighting a losing battle. Stop complaining and start contributing!

Personally, I watch the Wikipedia edits rolling up my screen, seeing contributions as they happen from individuals I’ll never know and am filled with optimism. Each edit is underwritten by a Creative Commons license which protects and preserves this body of knowledge for perpetuity. If there were world heritage sites on the Internet, Wikipedia would surely be the first to be recognised as such.

PubSubHubbub: Realtime RSS and Atom Feeds

Joss Winn / / Fun, Open Source, Standards & Specs, Twitter, Web

It’s made Dave Winer happy, which is no easy task, so I think PubSubHubbub is worth mentioning here. If it’s working as it should, this post should appear in my Google Reader, almost immediately after I’ve published it. That’s because PubSubHubbub is “a simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom [and RSS].” My blog feed is managed by FeedBurner which has already implemented the new protocol, as has Google Reader FriendFeed. They should therefore ‘talk’ to each other in realtime. Watch the video and you’ll see how it works. It’s pretty straightforward. It just takes a company the size of Google to push it through to adoption. The engineers say they were using it like Instant Messaging the night before the demo, which says something about how responsive this is. Technically, it should be another challenge to Twitter in that it allows for a distributed method of near realtime communication.  I’d like to see that. I feel like an idiot communicating within the confines of  Twitter, sometimes.