Tag: Yahoo Pipes

Example OUseful mashup for Online Journalism students

Joss Winn / / Data, Hacks, Journalism

I co-teach Online Journalism for level three students with Bernie Russell and this week, Tony Hirst from the OU came to Lincoln to give his, now annual, data-driven journalism class. Bernie and I prep the students a few weeks beforehand and then Tony rolls in and packs as much into the class as he can, leaving me and Bernie to pick up the pieces 😉
We’re grateful for it.

Here are Tony’s slides from this week

If you’re a student struggling with the Wikipedia/Pipes/Google Maps exercise, here’s a working example that you can clone and work backwards through to understand how it works. It’s basically slide 6 of the presentation above.

UPDATE: What follows is broken because of changes to the Wikipedia source page structure, changes to Yahoo Pipes and changes to Google Docs. Trying to keep it working is a pain, so it will have to stay broken for now.

Start by looking at this Yahoo Pipe:

http://pipes.yahoo.com/joss_winn/oj3ddjwikipediamashup

When you’re signed into Yahoo Pipes, clone the pipe and then click on view source of that example above. You’ll see this:

Source of pipe

The CSV source is https://docs.google.com/spreadsheet/pub?key=0Arh4BnSV2XSIdG1aUmd2dlFkTjdwRjlnazdKTk5mckE&single=true&gid=0&range=A1%3AG138&output=csv

You can look at the spreadsheet that is pulling data in from Wikipedia here:

https://docs.google.com/spreadsheet/ccc?key=0Arh4BnSV2XSIdG1aUmd2dlFkTjdwRjlnazdKTk5mckE&usp=sharing

Note how I’ve fetched the CSV into Yahoo Pipes, defined the data I’m interested in, renamed two key attributes, renamed the title attribute to be ‘population’ and then used the location builder in a loop block to determine the geo-locations. Once that’s done, it runs in the Pipe like this:

Is this displaying correctly? I’ve found that embeds directly from Yahoo Pipes can be a bit flaky.

However, if you right click on the KML link and paste the KML link into the search box of Google maps, then you should see something like this:


View Larger Map

You can see that both Yahoo Pipes and Google Maps allow you to embed the map into any web page.

Give it a try and get in touch if you’re having trouble. Of course, we can talk about it in class, too.

Creating a PDF or eBook from an RSS feed

Joss Winn / / Hacks, Mashups, Tips, Web

This morning, I found myself on Baseline Scenario, a well-known site which discusses the economic crisis. I noticed that the authors of the site had laboured over producing a PDF version for each month of their archive, by copying and pasting to Word and producing a PDF. There’s a nicer way of doing this, I think. When you’ve done it once, it should take you no more than ten minutes to go through the whole process any other time.

  1. WordPress provides a way to filter content by date. In our example, we’ll grab the RSS feed from the first month of publications: http://baselinescenario.com/2008/09/feed The permalink structure is clear enough on WordPress. For Blogger, it’s nowhere near as intuitive.
  2. The feed will display the articles in descending date order. When you are reading the PDF or eBook version, you don’t want to read the last article first, as you would on the website. To reverse the order of the feed, use Yahoo Pipes (or for WordPress, see @mhawksey’s comment below). You can clone my example. If you’ve not used Yahoo Pipes before, don’t worry. You just need a Yahoo account. The example I give is as simple a pipe as you will see and should make sense as soon as you look at it.
  3. Once you’ve created the pipe of the feed in ascending order, save and run the pipe. Look for the RSS icon and copy the pipe’s RSS link, which should look like this: http://pipes.yahoo.com/pipes/pipe.run?_id=cb438b51b2819eb1f4f5ec6f10daf09e&_render=rss
  4. Next, go to FeedBooks. Sign up for an account if you don’t already have one. Now, we create a Newspaper.
  5. Click on News in the menu and then on Create a newspaper. Give it a name and tag it. In our example, we’ll call it Baseline Scenario Archive.
  6. Click on ‘Add a RSS feed’. Give it a name (in our case ‘September 2008’)  and paste your RSS feed into the box. Once it’s found and accepted your feed, click ‘Publish’.
  7. You can now click on the name of the specific feed and you’ll be presented with a page that offers an ePub, Kindle and PDF versions of your feed. Here’s the Baseline Scenario September 2008 example.
  8. That’s it. You can do it with whole sites, too, if you like. Here’s one I did earlier (Blogger). The only thing you need to remember is to ensure that the RSS feed contains all the items you’re looking for. For the Blogger site, the source feed looks like this: http://www.blogger.com/feeds/27481991/posts/default?max-results=1000 A thousand items is more than enough to capture this site for quite some time. For WordPress, the site owner has to change their Reading Settings to include sufficient items. For the Baseline Scenario, they need to set this at a number high enough to ensure that a month’s worth of posts are included. I would just set it at 3000 and then forget about it. It would mean the entire site could be captured this way for the next year or so.

Having problems? Got a question? Leave a comment.