Thursday, 14 April 2011

Usability evaluation of the Futurelab web site

The site downloaded quickly, opening even the big banner graphic fast (I’m sure that’s something they thought about!). The ‘educationeye’ element of the page took a few seconds longer to fill with its content. All links seemed to work and pictures displayed fine. I didn’t come across any audio or video elements, which surprised me a little as I'm so used to seeing embedded video on educational resource sites these days.

The Futurelab web site includes a mini ‘educationeye’ on the front page, but I couldn’t understand why the content was arranged as it was. Education Eye offers “a way to discover and explore new ideas by mapping 100s of the top educational websites, blogs, forums and practitioner case studies.” The list view was obvious, but the ‘full’ view positioned items around the space using some metric I couldn’t fathom.

Clicking on the title took me to the full version of EducationEye ( which I found took quite a bit longer to load. It has a fancy interface which allows you to scroll around and view articles relating to innovation in education. I’ve been back to look at this site a few times as it feels like it ought to be a really useful site, but the interface gets in the way for me. I feel like I don’t know what I’m meant to do with it. There is the option to subscribe via RSS, which seems more attractive as I’d be able to scan the incoming feed headlines quickly, but I feel like I’m missing something and being incredibly dense and failing to appreciate whatever brilliant design they’ve come up with.

The site initially appears easy to navigate with the front page displaying a large menu of 4 aspects:

When you click on one of these, you get an expanded list:
However, this annoyed me, as I wasn’t able to see the detail of what was under each menu item until I'd clicked on the front page link - and then waited for the page to re-draw. Maybe my broadband connection got busier while I was using the site, but it certainly took longer to load these pages than it had the home page.

After quite a bit of fiddling, I found that the largest amount of useful looking content seemed to be on the Our Work and Resources pages. And then I discovered that the Resources page had a nice ‘Guided Search’ (effectively search by tag) feature... why wasn’t this on the home page?!

In terms of accessibility, I don’t know how a screen-reader would cope with this site. Perhaps the very simple top level of the menu would be easy for a screen-reader to interpret, but I don’t know how they work. I also don’t know how they work with Flash, but the interactivity provided by educationeye seems likely to pose problems for those with visual impairments.

I used the WAVE free web accessibility evaluation tool provided by WebAIM to check the site. WAVE doesn’t provide a complex technical report, but presents the original web page with overlayed icons and indicators that reveal the accessibility of that page. The outcome for Futurelab’s site seemed pretty favourable.
While this site deals with education, supporting teachers and providing resources, it is not intended to be a learning activity. Therefore, I didn’t really feel it was appropriate to evaluate in terms of pedagogical issues, learning design, sequencing, etc.

Context-specificGiven the intended audience, it might be argued that this site is designed for the technically confident, IT literate user, happy to use the tools available, knowledgable in search techniques, and able to easily recognise the web 2.0 features such as a blog and RSS feed. Conversely, the site could be intended to be for all those educators out there who really want to improve their innovative use of technology in learning, but who are nervous or lacking experience. I think that once they’ve found the Resources page they may be happy, but I feel that the navigation might put some people off - there is often a very real attitude (or at least learned behaviour) that if an item isn’t clickable from the front page, people don’t/won’t go digging for it. This makes web design a real challenge - providing readily available content, without completely cluttering up the front page.

Personal reflectionI have a niggling gut feeling with this site that, while it has good content and I think what Futurelab do is really interesting, the site itself feels like an example of design over substance. I feel that I shouldn’t be saying this... that they are a site focusing on technology in education, they know about design, accessibility, usability, they employ expert designers, and so it must be me who is wrong in not getting on with this site as well as I want to... but maybe, just maybe I’m not the only one?!

Wednesday, 13 April 2011

Reflecting on web 2.0 week

Web 2.0 is so last week! Well, it is if you're on H807 and already being encouraged to move on to evaluating usability, which is the next theme.

I had a lot of fun looking at mashups (more than once!), RSS feeds, social bookmarking, social networking and assessment 2.0.

There are just a few niggling thoughts that I haven't recorded so far...
  • I had fun looking at all these different tools and technologies, as did I think my fellow students, but the fact that they can be included in a course in innovation really shows that despite all the hype, and the growing number of users, there is a long way to go until we reach either widespread acceptance, or toolset stability. In a space of a few short years  tools and trends have come and gone, as shown by this History of Social Media Infographic, and for many people such instability is disconcerting.
  • While there were plenty of papers telling me about how brilliant these web 2.0 tools could be for education, finding actual examples and case studies was much tricker. Mashups seem a great concept, but mixing data from multiple sources isn't straightforward, and while there were lots of obvious suggestions for using RSS feeds and social bookmarking in teaching and learning, it seemed that social networking and assessment 2.0 have much further to go in terms of uptake. 
  • We often come across statements like "young people expect to use networked technologies in their learning", and "the net generation use their social networks for learning and expect their educators to do so too". I think a lot of caution needs to be exercised here. Steve Wheeler warns against accepting wholesale the idea of digital natives and immigrants, and advocates Dave White's alternative theory of visitors and residents. This all reminds me of the excellent Information Behaviour of the Researcher of the Future study by the British Library and JISC. It looks at the following claims about the Google Generation, and presents information which indicates that some of these are myths:
    • Google Generation show a preference for visual information over text 
    • Google Generation want a variety of learning experiences 
    • Google Generation Have shifted decisively to digital forms of communication 
    • Google Generation 'Multitask’
    • Google Generation are impatient and have zero tolerance for delay 
    • Google Generation find their peers more credible as a source of information than authority figures 
    • Google Generation need to feel constantly connected to the web 
    • Google Generation learn by doing rather than knowing 
    • Google Generation prefer quick information in the form of easily digested short chunks rather than full text
    • Google Generation have a poor understanding and lack of respect for intellectual property
    • Google Generation are format agnostic 
    • For the Google Generation, virtual reality may be as real as the real experience 
If you're short of time, pages 13-21 cover the claims and fact/myth findings.

Tuesday, 12 April 2011

Assessment 2.0

Assessment is the process of generating and gathering evidence of student learning, such that a judgement can be made about that evidence. Or at least it is in assessment 1.0. This is assessment as we have known it, and its characteristics have changed little in 100 years.

Web 2.0 covers "six big ideas" (Anderson, 2006, cited by Elliott, 2007):
  1. user-generated content
  2. the power of the crowd
  3. data on an epic scale
  4. architecture of participation
  5. network effects
  6. openness
Enter assessment 2.0. In a world where "web 2.0 is life 1.0 for most [younger] students" (Elliott, 2007), it is argued that the type of assessment activity best suited to the digital native will exhibit the following characteristics:
  • Authenticity - assessment of real-world skills and knowledge
  • Personalised - reflecting the knowledge, skills and interests of students
  • Negotiated - assessments may be agreed between teacher and learner
  • Problem-orientated - applying problem-solving skills to realistic tasks
  • Socially constructed - and involving the student's social networks in solving
  • Collaboratively produced - assessment is no longer purely individual and competitive, but recognises skills in collaboration
  • Recognising existing skills - and accrediting these.
Elliott describes assessment 2.0 in the following presentation:

The best site I found offering a collection of examples of assessment 2.0 was this wiki of case studies, from the "Web 2.0 Authoring Tools in Higher Education Learning and Teaching: New Directions for Assessment and Academic Integrity" project (part of the Australian Learning and Teaching Council).

It's a pretty up to date resourse, the case studies having been collected over 6 months in 2010, and include the following:
Social web activitySubject/DisciplineLevel of study
BloggingCultural StudiesUndergraduate

Cinema StudiesUndergraduate

Information ManagementPostgraduate


Media StudiesPostgraduate
Various Web 2.0 toolsEducationPostgraduate
Social networkingLanguagesUndergraduate
Photo sharingCommunication DesignUndergraduate

Virtual worldsLanguagesUndergraduate
Wiki writingAccountingPostgraduate




Information Technology (A)Postgraduate

Information Technology (B)Undergraduate


University College London's Learning Technology Support Service blog refers to a presentation by Prof Geoffrey Crisp which posts on examples of assessment 2.0, including:
  • Examine QuickTime VR image of a geological formation then answer questions based on that – drawing on things wouldn’t be able to see from static image.
  • Examine panograph (scrolling and zoomable image) of Bayeux Tapestry and answer questions drawing together different parts – students selecting evidence from different segments of the tapestry.
  • Interactive spreadsheets – Excel with macros. Students can change certain bits and answer questions on resulting trends in graphs. Can have nested response questions so that the answer to the second is based on first. (But there is a need for care with dependences so that a wrong move early on doesn’t lead to total failure).
  • Chemical structures using the Molinspiration tool. Students can draw molecular structures using the tool and copy and paste the resulting text string into answer which is held in the VLE quiz tool.
  • Problem solving using a tool called IMMEX (‘It Makes You Think’) which tracks how students approach problems. The tutor adds in real, redundant and false information that the students can draw on to solve the problem. They can use it all but the more failed attempts they make the fewer marks they get. We saw an archaeology example in which students had to date an artefact.
  • Role plays which can be done using regular VLE features such as announcements, discussion forums, wikis. Students adopt different personas and enter into discussion and debate through those personas.
  • Scenario based learning – this is more prescriptive than role play. The recommended tool is
  • Simulations – the site offers a virtual bank and factory. Students can work within bized then answer questions in the VLE.
  • Second Life (virtual world) assessment in which the avatar answers questions which go back into Moodle.
Examples of these and more are available through the site.

Other examples I came across include:

Elliott, B. (2007) Assessment 2.0 - Assessment in the age of Web 2.0. Available:

Sunday, 10 April 2011

Social networking

Once again this post investigates an aspect of web 2.0. I present a summary of some research and links on the use of social networking in higher education.

Facebook as a Functional Tool and Critical Resource
A course colleague posted a link to this a few days ago, and I just got a chance to read it. In a world where (some) university administrators consider use of Facebook to be subverting the official channels, Mark Lipton is integrating Facebook into his lectures in an innovative an inspirational way. He argues that since the students have Facebook on their laptops while they are in the lecture, he might as well take advantage of this! In his media studies course he is able to teach both about and through digital tools (like Facebook) and demonstrates and models and online identity to help students understand "responsible" Facebook use.

He creates Facebook class groups which are usually open because he insists on an approach to media learning that is open, social, and connected. Use of the Facebook group is not mandatory or graded. During lectures, a teaching assistant monitors the group wall and discussion lists. These are projected on the screen at regular intervals throughout any given lecture. Lipton argues that when students are given free reign without the stress of assessment, he can notice what they find important, where he needs to explicate, and when he should stop to give them voice to articulate their concerns

Lipton shares some of his lessons learned, which include:
  • sharing with students how he manages his Facebook identity - modelling responsible practice
  • in order to use Facebook—and not be used by it—one needs to understand how to operate its settings and options
  • he will not “friend” his students, but is happy to accept a student “friend request”
  • he makes explicit that any student/“friends” are added to a list that is used to block them from some of his more personal Facebook information, such as photo albums
  • he tries to ensure that students do not come up in his news feed and suggests that students block him from their news feed
  • he keeps communication with students as public as possible, asking students to post to the group wall so that everyone can participate, and avoiding the use of private Facebook chat.
Teaching with Social Networks: Establishing a Social Contract (pdf available here)

This paper describes the use of social contracts, collaboratively authored, updated and enforced by students in a blended class which describe permissible and expected behaviour in classroom and online contexts. This paper provides examples of successful student social contracts and describes students’ views on the impact of the social contract on their learning.

Facebook in the Language Classroom: Promises and Possibilities
This paper observes that e-learning tools have yet to be viewed as a mainstream component of foreign language teaching and have yet to become a foundational element used in L2 (second language) classes. It notes that "low level technology uses are generally associated with teacher-centered classrooms, whereas high-level technology usually promotes constructivist practices in which the students have to collaborate", but which require teachers to adapt new ways of communicating with students and to adopt new pedagogies. It argues that computer-mediated-communication can "positively modify teacher-centered models of interaction in the L2 classroom, and encourage students to interact with each other and rely on the L1 less as a consequence".

Facebook groups are identified as providing opportunities for L2 students to observe and participate in discussions from various regions of the world where the target language is spoken natively. It argues that opportunities for "intercultural communication with authentic native speakers of comparable age
language variation is of particular interest for intermediate and advanced language learners as it illustrates the richness of the L2 and introduces them to more authentic and colloquial language".

Despite all the hype, the web is not awash with case studies of integrating social networking into educational practice. While perhaps a little pessimistic (forgiveable, since it's from 3 years ago, but actually I wonder how much has changed...) I'll close with the following quote from George Siemens:
Social networking is still part of the hype cycle of educational technology tools. And for good reason. Involvement in a network can be a surprising waste of time…and a surprisingly effective way to learn. Social Networking in higher education looks at various common tools like Facebook and Twitter, and concludes “We’re incredibly excited about the things we can do in online and distance education with social networking…”As is often the case, the real story is where the action isn’t. It’s where the action will be. And I see that as the methods and approaches that we use to design curriculum, education, and our institutions. How long do we explore new tools and concepts until we are forced to consider the very spaces in which they occur?

Logos 2.0

LOGO2.0 part I
Image courtesy of Ludwig Gatzke, Creative Commons Attribution ShareAlike 2.0 License...
(and it's worth taking a look at the original of this image on Flikr where you can hover over the logos for more info).

Social bookmarking

The next tool in the investigations of web 2.0 tools and technologies is social bookmarking.

Sites like DeliciousDiigo, Clipmarks, and many more, allow users to collect internet resources and links, apply descriptive tags, and share them with particular groups of people or the public as a whole. Common Craft has a nice 3 minute introduction to social bookmarking in plain English. (Actually they have videos on a big list of internet and technology topics).

A quick run-down of the benefits of social bookmarking include:
  • Ease of use - access from any computer. Bookmarks are not saved in one browser/application
  • Public sharing - allowing you to find bookmarks recommended by others 
  • Resources which are perceived as more useful will be shared more, hence promoting or ranking links
  • Users define their own descriptive tags - no need to learn or be restricted by any imposed classification framework
  • Provides a means of sorting large collections of bookmarks, which can become unmanageable with traditional methods
  • Shared bookmarks may be rated, commented upon, auto-published to blogs and other media, annoted, subscribed to via RSS feeds.
And in the interest of balance, some downsides:
  • Idiosyncratic/non-standard use of keyword tagging - inconsistencies in tagging (e.g singular/plural, capitalisation, spelling errors)
  • Tags may have more than one meaning, or be used in unusual ways by some users
  • Tags do not support the definition of hierarchical relationships
  • Assumption of reduced personal privacy - although some sites allow private tagging
We were asked to consider an educational context in which social bookmarking might be appropriate and useful. Obvious examples include:
  • Allowing students to access a teacher's bookmark list
  • Enabling students to collaborate on a project by sharing the resources they find
  • Curating a social bookmark collection formed from other student-generated content that is produced during a course or project (e.g. wikis, blogs, podcasts, videos, mindmaps, documents, diagrams, etc etc)
  • Allowing teachers to find lesson resources produced by other teachers

I found a very interesting project (supported by the Higher Education Academy's Education Subject Centre) investigating whether it is possible to assess social bookmarking activities. The premise is that, because Diigo supports conversation, it might therefore support higher order cognitive activities. Students are helped to go beyond just ‘remembering’ (creating bookmarks) and move to ‘understanding’ (using tags), ‘evaluating’ (by offering comments) and even ‘creating’ (such as planning essays). Because these higher-level skills are involved, assessment becomes possible, A marking scheme can reflect both the cognitive and social nature of social bookmarking and consider:

  • the number and type of texts that a student bookmarks
  • the quality of commentary given for each bookmark (emphasizing meaning-making and critique above listing and tagging)
  • the level of interaction with peers.

More information on this interesting project in a blogpost. The author also has another blog post giving a useful list of recommendations on helping students make the most of social bookmarking.

Another interesting article (also linked by the author of the above study in face) is on the limitations of Delicious and how these could be used to encourage learner engagement.

Unlike the conversation aspects of Diigo featured in the project above, Delicious is sometimes criticised for not encouraging participation or community. This article argues that that 'weakness' might actually make it the right tool for those learners who struggle with active participation and collaboration. It suggests that collecting and sharing links is a more neutral activity than, say, contributing to a wiki and so might be helpful in reducing  anxiety associated with participating online.

I've not really used social bookmarking much before. I've had fleeting uses of StumbleUpon and Clipmarks, but not maintained use of any. I find that in normal web usage I rarely bookmark now - search is so improved that I can find what I want so easily. However, if you've forgotten something exists, then search might not help, so I am beginning to want to bookmark again, particularly for this course. In my PhD I maintained a TiddlyWiki, in which I saved links in as well as adding commentary and notes on papers I read. This is probably why I didn't need social bookmarking at the time. I'm doing something a little bit similar with this blog, but I think a tool designed more for the purpose of saving links will be useful. The Student learning with Diigo site offered the following two reasons which have persuaded me to try out Diigo at the moment:
  • Diigo allows users to annotate webpages. Other social bookmarking sites let users save a website title, description and to tag the website with relevant keywords. Diigo does this, but also lets users highlight portions of a webpage, or to add a virtual "sticky note" on the website. And the annotations can be made available to the entire network to help other users.
  • Diigo saves a screenshot of webpages. This can help users to remember what the page looked like previously, especially if the page was changed over time.

Thursday, 7 April 2011

Tools for assessing web accessibility

Kevin, a former H808 colleague, posted the following on accessibility today - very useful as next week on H807 we are investigating usability.

This post is also made automatically from my new account - a social bookmarking tool which I'm experimenting with as part of this week's web 2.0 tool investigations.
Amplify’d from
Filed under: Uncategorized — kevhickeyuk @ 9:54 am
Here are a few tools that should help anyone  who wants to make sure their website/intranet/ online learning content, is as accessible as possible;
  • The Accessibility colour wheel allows you to select foreground and background colours, simulates how they would look to those with different types of colour blindness.  If you select a colour combination that is good, in terms of accessibility, you will get a ‘good’ message appear on the screen.
  • WAVE is a free web accessibility evaluation tool provided by WebAIM. It is used to aid humans in the web accessibility evaluation process. Rather than providing a complex technical report, WAVE shows the original web page with embedded icons and indicators that reveal the accessibility of that page.
  • The Fujitsu Web Accessibility Inspector is one of a number of tools that will validate a webpage according to the Web Content Accessibility Guidelines (WCAG) guidelines.  This can produce a detailed report which identifies a number of issues.  If you are using a system such as SharePoint, then it might highlight issues that you will not be able to easily change.

More mashups

Really enjoying exploring mashups, in particular data visualisation. In the last couple of days I've come across these:

Describes itself as "A ninja tool for helping you house or job hunt within Great Britain". You put in your postcode, and it provides a map with a mask covering areas which you include or exclude on the basis of commuting distance, house price, and scenic-ness. It's currently in beta, but I signed up today and got my invite only hours later. It seems that you will soon get invites to give to friends too.

Pointed out by Kate, Dipity allows you to create interactive timelines. You can add events, pictures, blog feeds etc to a timeline, or you can add content from any RSS feed. Here's my timeline based on the BBC Technology RSS feed.

Google Public Data Explorer
Another offering from Google labs, they say: "The Google Public Data Explorer makes large datasets easy to explore, visualize and communicate. As the charts and maps animate over time, the changes in the world become easier to understand. You don't have to be a data expert to navigate between different views, make your own comparisons, and share your findings."

For example, this one tracking life expectancy at birth against fertility rate:

Wednesday, 6 April 2011

RSS feeds

Next on the list of this week's web 2.0 tools is RSS - Really Simple Syndication, or Rich Site Summary, depending who you ask. has a nice one page summary of what RSS is.

Essentially, RSS allows people to subscribe to a 'feed' from a chosen web site, blog, search result etc. They do this using a feed reader or aggregator, which automatically collects any updates from the chosen feed, and means the user only has to check one place in order to follow updates from multiple sites.


  • RSS feeds can be 'delivered' in the preferred medium, e.g. by email, in an aggregator/feed reader, or in a widget embedded on another page (such as this H807 blogs page
  • RSS is opt-in only - you get updates only from the site feed, and don't end up with spam or un-requested content
  • Convenience: you only check the feeds when you want to - and don't need to worry about missing something if you don't check for a while, because it will be picked up automatically by your aggregator
  • Time saved by not having to visit multiple sites in order to get or check for updates
  • RSS can be used to monitor news tickers, and search engine outputs, as well as blogs and web pages - useful for monitoring the occurrence of a particular key word or phrase in a news article (the #H807 Twitter feed at the bottom of the left column of this blog is an RSS feed on the search term "#H807")
  • Having my feeds all in Google Reader makes it really easy for me to 'favourite' particular posts or stories - and save them for future reference
  • I find I often now subscribe to an RSS feed rather than bookmarking a site. This way I don't have to remember to check it
  • Readers/aggregators tend to offer a 'headline' or 'first line of post' type display - that allows you to quickly scan feeds in order to decide whether to read the full content
  • I love having the RSS on my own blog so I can track what is getting hit, how many users and where they come from. All the stats are geeky and far more absorbing than I should let them be!

  • Subscription means you only receive feeds from sites which you know about - no serendipitous discoveries (although in actual fact the culture of linking to other pages from a blog or other news article means you may well discover new links of interest)
  • A feed from a particularly busy source might leave users feeling overwhelmed... but they are free to unsubscribe!
  • Not all content is suitable for RSS
  • It's still geeky - it's not on the radar of the internet using masses who have mastered email and browsing, but not yet caught up with this type of technology
All my feeds coming in to my Google Reader, and also showing up in a list on my iGoogle page (which is open in one of my browser tabs almost all the time I am online) mean I see plenty of updates within minutes. This is brilliant for things I need to read, but a lot of the time it takes a certain amount of self-discipline not to reed all these exciting feeds the minute they arrive... and sometimes I have it, and sometimes I don't!

Monday, 4 April 2011

The Granny Cloud

Quick post on a lovely elearning innovation I heard on BBC Radio 4 Click On this evening.

'Grannies' (in fact male or female, and not necessarily 'elders') are using Skype to link up with self-organising learning groups of children in deprived areas of India. The Grannies are not teachers as such, but help facilitate informal classes with a group of incredibly learning-hungry children.

Also a great TED talk on the same:


This week is looking at lots of web 2.0 tools. First up are Mashups.

The concept of a mashup is the creation of something new from parts which come from various separate sources, resulting in an integrated whole. Many of the examples illustrate the integration of information with maps, providing easy to use visualisation of data.

This reminded me of a couple of things which I've seen recently, including a map of Christchurch earthquake-damaged buildings due for demolition:

View Christchurch Building Status: April 2nd 2011 in a larger map

and the Nerdy Day Trips map begun by Ben Goldacre and friends. I also have my own example of geo-tagging my photographs on a walk around Toronto.

These are nice examples because they also illustrate that a map mashup may be created by one person, using an official data source (as in the Christchurch map), by adding personal digital content (my photos) to a public source (Google map)  or may be a massive collaborative effort with many contributors (as in the Nerdy Day Trips).

We were asked to read Brian Lamb's ‘Dr. Mashup; or, Why Educators Should Learn to Stop Worrying and Love the Remix’, which I found a bit of a struggle. It was quite an interesting article, but longer than I needed - I'm sure there is content in there which would be interesting to return to when I have a bit more time.

Lamb distinguishes between content remixes/mashups (a combination of two or more works) and data mashups, which "combine data and functionalities from two or more Web applications". I think I've previously only had the idea of data mashup, so hadn't really considered that some might suggest simply combining content is also a mashup. If that is the case then the iGoogle page is a nice example, giving you the opportunity to aggregate lots of content (RSS feeds, email, social media links, calendar etc) on one page. As I think about it more, much of my iGoogle page relies on RSS feeds (from my feeds to PhD and xkcd cartoons, to my Google Reader list, to New Scientist headlines) and some of the content I've added probably is a mashup in its own right, for example the live train times from my local station gadget, which uses the API provided by National Rail Enquiries.

Kevin helpfully pasted an article which I much preferred, This describes mashups by integration and mashups by aggregation, which is effectively the same distinction as given by Lamb, but for some reason I absorbed it more easily - and I'd already been thinking about aggregating content too!

A few more examples of nice mashups I found:

  • Ordnance Survey map overlays for Google Earth, produced using the Ordnance Survey OpenSpace API
  • Audio/video mashups such as Wesch's 'Rethinking Education':


Brian Lamb, B. (2007) ‘Dr. Mashup; or, Why Educators Should Learn to Stop Worrying and Love the Remix’.