Since 2010 a project called Student as Producer has been adopted as the de facto teaching and learning strategy for the University of Lincoln. This is an attempt to engage undergraduate students in research, and to make research part of the teaching process. It is also a vehicle for demonstrating the value of openness – an idea bolstered by the establishment of numerous other open access initiatives at the university.
One of the lasting outcomes of our Total Recal ‘rapid innovation’ project in 2010, was that Alex Bilbie wrote the first (and only) OAuth 2.0 server for the CodeIgniter PHP development framework that we use. Since then, he’s been refining it and with every new project, we’ve been using it as part of our API-driven approach to development. As far as we know, the use of the OAuth 2.0 specification, which should be finalised at a forthcoming IETF meeting, is not yet being used by any other university in the UK. There are a few examples of OAuth revision A in use, but OAuth 2.0 is a major revision currently in its 23rd draft.
As a result of his work, Alex was invited to talk about OAuth 2.0 at Eduserv’s Federated Access Management conference last year.
Nick Jackson gave the same presentation at the Dev8D conference a couple of weeks ago.
Since Total Recal, we’ve used OAuth 2.0 for Jerome, data.lincoln.ac.uk, Zendesk, Get Satisfaction, and more recently Orbital and now ON Course. We’re at the stage where our ‘single sign on’ domain https://sso.lincoln.ac.uk is the gateway to our OAuth 2.0 implementation and it will soon be running on two servers for redundancy. In short, due to various JISC projects helping pave the way, it has been formally adopted by central ICT Services, and staff and students are gradually being given control over what services their identity is bound to and what permissions those services have.
The work Nick is doing on the Orbital project is extending Alex’s OAuth 2.0 server to include some of the optional parts of the specification which we’ve not been using at Lincoln, such as refresh tokens and using HTTP Authentication with the client credentials flow. This means that the server is able to drop straight in to a wider range of projects and services.
Recently, JISC published a call for project proposal around Access and Identity Management (AIM), which I am starting to write a bid for. Appendix E1 states:
JISC is particularly interested in seeing innovative and new uses for OAuth. Bids should show how this technology brings benefits to the community and can help address institutional requirements within research, teaching and learning, work based learning, administration and Business Community Engagement.
In Total Recal, we released version 1 of the server code but have learned a lot since that project through integrating OAuth with other services. Version 2 of our OAuth server is more representative of our current implementation and fully implements the latest draft (23) of the specification.
However, this is what access and identity management currently looks like:
At the moment, the most widespread use of the OAuth server is Zendesk, our ICT and Estates online support service. Projects such as Jerome, Orbital, and ON Course, as well as three 3rd year Computer Science student dissertation projects are using it, too. The plan is to use OAuth alongside Microsoft’s Unified Access Gateway (UAG), which can talk SAML to OAuth via the OAuth SAML 2.0 specification. Here’s what we intend to do:
The primary driver for this is the ‘student experience’ and it cuts three ways:
Richer sharing of data between applications: A student or lecturer should be able to identity themselves to multiple applications and approve access to the sharing of personal data between those applications.
A consistent user experience: What we’re aiming for at first is not strictly ‘single sign on’, but rather ‘consistent sign on’, where the user is presented with a consistent UX when signing into disparate applications.
Rapid deployment: New applications that we develop or purchase should be easier to implement, plugging into either OAuth or the UAG and immediately benefiting from 1) and 2).
Following a recent meeting between ICT and the Library, we agreed to take the following steps:
All library (and ICT) applications that we operate internally must have Active Directory sign-in instead of local databases. Almost all of our applications achieve this already. This is the first step towards step (3).
All web-based applications must offer a consistent looking sign-in screen based on the sso.lincoln.ac.uk design (which uses the Common Web Design). This is the second step towards (3).
All systems must implement web-based single sign on via OAuth, SAML or ADFS and they will be sent to either UAG or the OAuth/SAML server.
The library are going to investigate to what extent we can do (2) with their applications such as Horizon and EPrints, and from then on, systems that are purchased or updated must do (3). It also makes sense to look at EPrints and WordPress in the short-term as applications that can use OAuth.
Two of the outputs we’ll propose to JISC are a case study of this work, as well as further development of the open source server Alex and Nick have been developing including an implementation of the OAuth SAML specification that we’ll share. Like our related work on staff profiles, the need to get access and identity right is becoming increasingly apparent as staff and students become accustomed to the way access and identity works elsewhere on the web. For Lincoln, a combination of OAuth and UAG is the preferred route to achieving consistent sign on across all applications, bridging both the internally facing business applications managed by ICT (e.g. Sharepoint, Exchange, Blackboard) and the more outward facing academic and social applications such as those developed and run by the Library and the Centre for Educational Research and Development.
During my attendance at Dev8D last week, I used the time to start pulling together some ideas I’ve been knocking around for a few months now. Most of the projects I’ve helped set up or led over the last three years have involved working closely with developers working in higher education and ever since JISCPress, we’ve been growing a culture where student developers, either on bursaries, employed part-time or full-time recent graduates (i.e. their first ‘proper job’) are core contributors to the project. My point here is that through these projects I have been able to observe how young hackers learn their craft and make the transition from a formal education in Computer Science to learning on the job (i.e. apprenticeship).
I use the word ‘craft’ deliberately. I think our work is much more closely aligned with craftsmanship than with engineering but I was unsure how to articulate this until I read Pete McBreen’s Software Craftsmanship. McBreen argues that a focus on craftsmanship is to return to the roots of software development:
Good software developers have always understood that programming is a craft skill. Regardless of the amount of arcane and detailed technical knowledge that a person has, in the end, application development comes down to feel and experience.
McBreen distinguished software craftsmanship from software engineering and computer science, not as their opposites but as a different tradition “that happily coexists with and benefits from science and engineering.” He compares the software craftsman to the blacksmith, both of whom transcend science and engineering and benefit from improvements in their tools, materials and understanding. For McBreen, GNU/Linux is an example of software craftsmanship that thrives due to the dedication, skill and craft of the people who contribute to the development of the operating system.
‘Software engineering’ was born out of a so-called ‘software crisis’, identified at a NATO conference in 1968. It was determined that the way out of that crisis was to apply an engineering approach to large scale state of the art defence projects. Since that time, argues McBreen, “the needs of the US Department of Defence have dominated the conversation about software engineering.”
software engineering is not all that relevant to many projects. Software engineering was created to solve the problems of really large groups working on multiyear projects. Most modern software development, however, is done in relatively small teams.
Not surprisingly, the practise of software craftsmanship is often allied with agile development, which emphasises the human aspects of software development, the creative and variable processes involved in creating software that meets human needs. Both share a concern with quality and both encourage continual reflective critique of one’s work in order to improve the software and oneself. In his book on Crystal Clear, for example, Alistair Cockburn identifies ‘reflective improvement’ and ‘osmotic communication’ as two of the three minimum requirements for a Crystal Clear project. In practice, reflective improvement refers to workshops and end-of-iteration discussion about how the work is going and how it might be improved. Osmotic communication refers to how flows of information between developers should be unobstructed by locked doors, walls and corridors. It’s about sitting developers in the same or adjacent rooms so that they absorb information about the project with as few barriers as possible. Access to information, a safe environment where people can respectfully speak their minds and close collegial working are highly valued. At times, developers will program side-by-side or in pairs at one workstation, so as to review each others’ work. He refers to this as ‘peer code peering’.
Both reflective improvement and osmotic communication can be enhanced by the choice of tools developers use and the feedback they provide to her. Craftsmen in all trades rely on their tools to provide feedback – a joiner’s plane, a responsive power drill, etc. Many craftsmen will make their own tools to improve their responsiveness. Cockburn notes that one of the most important tools developers can use is an automated regression test suite, which allows the team to continuously test their work and provides feedback to each developer about the quality of their code. Feedback from a Continuous Integration (CI) suite of tools can be usefully presented by ‘information radiators’, dashboards which typically provide status information of servers, the number of use cases delivered and the number of tests passed. Although, Cockburn doesn’t use the term, I think that reflective improvement and osmotic communication refer to the ‘learning environment’ that the software craftsman creates so as to improve their understanding of their work and further develop their craft.
References to the importance of learning from others and from one’s work are made throughout McBreen’s book, as well as an entire chapter at the end of the book called ‘Perpetual Learning’. There, he outlines how to create a ‘learning environment’ by building a library of books for developers to read, as well as ensuring that they take time out each week to practice or learn something new. Like Cockburn, he emphasises the importance of workshops and a series of seminars where developers discuss their work. In addition, McBreen suggests that developers are encouraged to attend and present at conferences and write papers about their work as well as take on the role of instructor where they are able to do so. Craftsmen continually undertake self-directed learning, preferring non-proprietary, open source tools that are sustained by a community and made freely available to learn from, but more importantly, software craftsmen learn from each other, with master craftsmen mentoring journeymen and apprentices.
In a more recent book on software craftsmanship, Apprenticeship Patterns, Hoover and Oshineye also devote a chapter to ‘Perpetual Learning‘, offering further practical advice to aspiring software craftsmen. They list the following:
Expand your bandwidth: Read books and articles, engage with your peers via conferences, social media and mailing lists, join user groups, study from open educational materials on the web
Practice, practice, practice: Take the time to practice your craft without interruptions, in an environment where you can feel comfortable making mistakes. i.e. ‘deliberate practice’ They borrow the term ‘code katas’, whereby programming exercises are repeated again and again until they become ingrained in the individual.
Breakable toys: Budget for failure by designing and building toy systems that are similar in toolset, but not in scope to the systems you build at work. i.e. build something personal to you, that you will learn from. Develop your own CMS or game that you can afford to break while learning.
Use the source: Seek out other people’s code and read it. Start with the applications and tools you use every day.
Reflect as you work: Become a reflective practitioner of software development. This involves regular introspection into how you are working.
Record what you learn: Keep a record of your journey in a journal, personal wiki, or blog. A chronological record of the lessons you have learned can provide inspiration to those you mentor, since it makes your journey explicit, but it can also give you a vital resource to draw upon.
Share what you learn: Early in your apprenticeship, develop the habit of regularly sharing the lessons you have learned. i.e. keep a blog, run workshop sessions, be part of a community of learners.
Create feedback loops: Create mechanisms for regularly gathering more or less objective external data about your performance. i.e. automated regression tests, information radiators, exams and certifications, pair programming, and asking your peers what they think.
Learn how you fail: Seek to identify the ways in which you tend to fail and try to resolve those that are worth fixing.
In his book, The Craftsman, Richard Sennett chooses to focus on open source software development as a modern form of craftsmanship (“the skill of making things well”). The value of Sennett’s book is its breadth of scope. While making no reference to McBreen’s earlier book, Sennett situates open source hacking and the development of GNU/Linux within the social and economic history of craftsmanship and our relationship with technology. For Sennett, Linux is a “public craft” and open source hackers are a “community of craftsmen.” In terms of learning this public craft, he compares ancient pottery making with open source hacking and finds that only the speed between problem finding and problem solving differentiates the two. In programming, and especially open source programming, the velocity at which we can learn can be much greater than in traditional, material crafts. Our tools and the open, distributed nature of developer communities enhance our opportunities for learning. ((Sennett’s observations also help us consider software craftsmanship together with learning theories such as cognitive apprenticeship, situated learning, and constructivism.))
In The Nature and Art of Workmanship, David Pye does not make reference to software development but does develop a very useful framework for identifying and understanding craftsmanship, which complements much of what Sennett and McBreen describe. For Pye, craftsmanship is the workmanship of risk; that is, work that is constantly at risk of error in the process of creation. A simple example of this is writing with a pen. In contrast to the workmanship of risk (craftsmanship) is the workmanship of certainty; that is, workmanship where the quality is always pre-determined and is usually found in quantity production, and always in fully automated work. An example of this would be modern printing. The workmanship of certainty is most common in modern, industrial society, but has existed in some form for hundreds, if not thousands of years. All types of workmanship exist somewhere on the axis between risk and certainty and furthermore are subject to varying degrees of regulation or freedom.
What distinguishes the degree of risk or certainty for Pye is the extent to which the workman’s tools regulate his work. Pye argues that a pure form of workmanship of risk is hardly ever seen in any trade; for centuries people have developed tools to help regulate their work in some way (e.g. jigs) and guarantee some degree of certainty in the quality of their work. Regulation of work does not necessarily lead to certainty as some tools such as a lathe can be used in combination with a free hand, producing unique objects that are nevertheless regulated in some respects such as their size but not their form.
A simple way of asking whether it is workmanship of risk or certainty is to ask, “is the result predetermined and unalterable once production begins?”
In the drawing above, I have speculated that software craftsmanship as McBreen describes it, would be called workmanship of risk by Pye, with regulation introduced by tools such as Integrated Development Environments (IDE), Continuous Integration (CI) suites and the general operating system environment the developer is working with. Although regulated, like almost all workmanship of risk, the software craftsman often produces something bespoke to the users’ requirements, iteratively working towards the desired result through the writing and refactoring of code. Agile software development recognises that the result of most software development cannot be predetermined and that projects run best when they remain responsive to the users’ feedback. Software Engineering, on the other hand, aims to eliminate as much of the risk as possible and pre-determine the outcome of the design and programming effort, which both McBreen and Pyritz both identify as a form of Taylorism.
The Institute of Electrical and Electronics Engineers defines software engineering as “the application of a systematic, disciplined, quantifiable approach to development, operation, and maintenance of software”. Pyritz questions whether the Taylorist model of scientific management is even viable in software development, quoting Humphrey, who asked ‘Why don’t they practice what they preach?’
The general practices of industrial software engineers are poor by almost any measure”. Why? “The educational system does not provide graduates with the practical skills they will later need. . . . Few software organizations are willing or able to provide the remedial training their new engineers need. Today’s software organizations have few if any role models who consistently demonstrate effective work habits and disciplines.
In learning to become software craftsmen we need role models, too. Both McBreen and Sennett emphasise the importance of apprenticeship and slowly learning by doing with others. This is what JISC’s DevCSI offers to the sector, running regular hack days, DevXS and the annual Dev8D conference. This year over 250 developers attended Dev8D to learn from each other.
While I was at Dev8D, I issued an informal survey asking developers a number of questions about their working environment, how they learn their craft, the tools they use and an assessment of their skills. I shall provide a summary of the responses in another post [UPDATE: here it is], but I am encouraged to do further research in determining how developers (working in tertiary education) learn their craft and how opportunities for learning might be improved.
Another related project I started while at Dev8D was Hacking the University, a simple website intended to collect short interviews with developers working in universities. It is inspired by The Setup and I hope that over time, it will provide a record of the people working in this community and add to the recognition of the work they do, how they learn and how their working environment impacts upon their work and learning.
If you are a developer working in a university, please consider contributing to Hacking the University and telling others about your approach to your craft.
Just a heads up to say that we’ll be advertising for a Web Developer to work on Orbital, our JISC-funded ‘Managing Research Data’ project. The post, starting in March/April, will be a 12 month, full-time, grade 5 (c.£21K) position.
The Web Developer (‘you’) will be working in the Centre for Educational Research and Development, alongside Nick Jackson, Lead Developer on Orbital, and also benefit from being in a team that includes staff in central ICT services and the Library. Orbital builds on and extends previous work we’ve been doing over the last couple of years, so if you’re interested, you should read through our projects pages.
If we were to summarise our technologies and interests I guess they would be #agile, #opensource, #opendata #LAMP, #php, #codeigniter, #mongoDB, #OAuth, #APIs, #HTML5, #CSS3, #github and moving towards #RDF and #LinkedData.
Just seeing these hashtags listed together should cause your heart to beat with excitement 🙂
When we advertise in January, you’ll see that the job spec is actually a pretty standard affair. What I want to emphasise here is how interesting and fun the job will be.
The key section in the Job Description is what you’d be working on with Nick:
Development and implementation of a set of web services, which re-use and develop our previous, JISC-funded work as well as other initiatives (e.g. SWORD and DataCite DOIs).
Documented source code will be made available under an open source license by the end of the project.
Development and implementation of mechanisms for managing and transferring data, including the use of MongoDB, OAuth, read/write RESTful APIs, SWORD2 interoperability, and integration with the administrative functions of EPrints.
That actually summarises a lot of work.
I’m managing the project and try to run things with as little hierarchy as possible within a university environment. You’ll always know the project priorities and will be trusted to self-organise and deliver on time, working to two-week iterations and, roughly, monthly releases. I regularly reflect on how we work and our overall working environment. For Orbital, I favour the Crystal Clear agile methodology, as does Nick. You’ll be encouraged to reflect on this with us, too.
We work hard, and not always 9-5pm, but we work at a pace that is sustainable over a long period of time. We take our work seriously but, in the spirit of hacking, are always looking for ways to have fun, too. We recognise that we’re fortunate to be working in a diverse and intellectually stimulating academic environment, but are user/product focused at the end of the day. You’ll be working directly with our users, who are Researchers in the School of Engineering and Siemens, and staff in the Library and ICT. You’ll need to be showing them refreshed, working software every couple of weeks and iteratively improving Orbital, based on their feedback and requirements. There may also be times when you’ll be asked to talk publicly about your work and you’ll be encouraged to blog about it every so often, too. I expect the project to produce one or two conference/journal papers, and you’ll be named as a contributor and can take as active role in that as you like.
I hope this sounds like an interesting job. At £21K, I recognise that it will probably attract younger developers looking to gain experience, though of course, we welcome applications from anyone whatever your age. By the time the post starts, we’ll have set up a decent dev/staging/production environment, hosted in the cloud, and relying on Github and Jenkins to keep things versioned, integrated and tested. Nick will have been developing Orbital for a couple of months or more and laid the groundwork for someone to start coding quickly in a supportive environment.
If you’re thinking of applying and don’t live in Lincoln, you’ll be pleased to know that it’s a decent small city, and a relatively cheap place to live. The campus is modern and sits by a Marina in the middle of the city. You can walk to work. I love the place. Oh, and you can choose your own hardware for development, within reason. Most of us use Macs, but whatever suits you. I’ll ask the successful candidate what they prefer when we offer them the job.
If, after reading around the project website, you’ve got any questions about the post, please do get in touch. Thanks.
* Wondering what the hell ‘web scale’ means? Something like this.
I’ve spent the last five months helping to organise and host DevXS, a national student developer conference. The conference on 11-13th November was fully booked and a great success. Over 170 students attended from across the UK, representing 37 universities, as well as a further 20 tutors and developer mentors working in the Higher Education sector.
You can read more about DevXS on the conference blog which was updated throughout the weekend by a superb team of media students. There are lots of videos and presentation slides on the blog as well as pictures and information about the prize winners and their applications.
It was a really exhausting and satisfying experience to be involved in and not only was it the first conference of its kind in the UK but it looks like it will become an annual event hosted by a different university each year and organised by the JISC-funded DevCSI project.
You can read a report about the conference on the DevCSI website. The Guardian also published an article (originally titled ‘Hacking the Academy’) in the run up to the conference, which I wrote with Mike Neary.
Google gave us Knol (we didn’t need it), now it’s taking it away (meh…), and instead has given us Annotum, something far more useful than Knol was ever going to be. Annotum is a free (as in speech and beer) theme for WordPress that turns a site into an open access publishing platform suited to scholarly journals. It’s more than just a theme though. It comes with several built-in plugins, too, which extend the functionality of WordPress significantly, including the introduction of a new workflow based around peer-review.
When I first read about Annotum this morning, I was both pleased and disappointed. Pleased to see that finally there was a decent attempt to turn WordPress into a journal publishing platform and disappointed that it didn’t come from the academic community itself. We’ve seen the utility of WordPress for scholarly publishing for a few years but never concentrated our efforts enough to actually deliver what Google and partners have done (significantly, the lead Developer, Carl Leubsdorf). Thankfully, Annotum is not only an open source theme for WordPress (and, perhaps more significantly, available to use on WordPress.com right now), but development is also taking place on Github, which is about the most collaborative development environment there is. It’s begging for academic input from developers, publishers, users and tinkerers like myself, with no barriers to participation from what I can see. I’ve made a couple of suggestions to start with. There’s a number of people doing work on scholarly publishing (e.g. #jiscPub, #scholarlyHTML) and I’m sure their work would make Annotum better for all of us.
Yesterday I had another academic ask me if I could help them set up a scholarly journal and so I pointed out the OJS system that I run. To be honest, while OJS is the gold standard for blind peer-reviewed open access scholarly journal publishing, the barriers to entry are high and, well, I agree with this guy. As of today, I’ll be recommending that we look at Annotum before jumping to OJS. I’m pretty sure the response will be positive from my colleagues.
I’m going to the JISC Information Programme Meeting on Thursday and have been asked to join a panel where I’ll talk about our work at Lincoln under the heading ‘from unprojects to services’. Here are my notes.
Over the last couple of years, staff in CERD, The Library and ICT have worked closely together on a number of ‘rapid innovation’ projects, which have sometimes later attracted JISC funding. Much of our work has been undertaken at the initiative of individual staff, who have benefited from a supportive ICT environment that allows us the freedom to develop and test our ideas without running into bureaucratic walls. ICT – in particular the head of the department, Mike Day, and head of the Online Services Team, Tim Simmonds – recognised the benefits of employing undergraduate students and recent graduates, and established a post which Nick Jackson and Alex Bilbie share. Alongside this, I have been applying for JISC funding and successful bids have allowed us to employ Nick and Alex full-time rather than part-time. In recent months, this has worked very well and currently much of their time is spent working on JISC-funded projects which bring value to the University. Below, are a list of the services that this culture of innovation has allowed us to work on over the last year or so. Click on the links to go to the services.
Most of these services push and pull data to Nucleus, the central, open datastore built on MongoDB. e.g. Zen Desk=People + Locations, My Calendar=Events, Jerome=Bibliographic
We’re currently looking at how Nucleus can also be a source for Linked Data. It has open(ish) APIs.
CWD sites transparently sign the person in to the site, if they are signed in elsewhere.
We like Open Source. SSO is mostly open source software. Alex has released his OAuth 2.0 code. CWD likely to be open source; MongoDB, bits and pieces from Jerome and My Calendar.
As we build these services, they are being integrated, too. e.g. lncn.eu will be a URL resolver for Jerome offering realtime monitoring; posters will show up in My Calendar events; CWD is the design framework for My Calendar.
Most of these services are for official launch in September. They will be included in the new ICT Handbook, included in brochures and other announcements.
Now that we know we can develop this way and that it works and we enjoy it, we’re hoping to expand from two to four student/graduate developers and have our own budget for hardware/software/conferences and to give to staff and students that want to join us.
Our approach links into the University’s Teaching and Learning Strategy: Student as Producer. We want to work with students and staff across disciplines to create useful, innovative and enjoyable online services that make the University of Lincoln a great place to work and study at. It’s not about a team that works on ‘educational technology’, but rather a network of people who develop and support technologies that make Lincoln a productive environment for research, teaching and learning. It’s inclusive, with students (and therefore learning) at its core.