Innovative practice with elearning - reflections

A few reflections on the case study reading...

Some of these case studies do not sound particularly inspirational. While they undoubtedly had tangible benefits, were they innovative? Again, back to the question of what it means for elearning to be innovative. Well, a number of these practices (providing resources online, developing podcasts, computer-based assessment, etc) sound like they are old hat. However, given an analysis of actual current practice in HE institutions (all these case studies were HE), I wonder whether we really would find a prevalence of these elearning practices. More likely, I suspect, that while there is greater awareness of the tools and technologies and the potential of elearning, and that the technological barriers are ever decreasing, perhaps many of these practices have not yet made the transition to mainstream. In that case, looking back on these case studies of 5-7 years ago, perhaps they really were innovative. Also the fact that we now accept that practices such as those described seem familiar, it suggests that they are being adopted and integrated into wider practice - and surely that must be test of whether an innovation really makes a useful difference.

I’m struggling a bit to think how these case studies can relate to my own context - a technology consulting company with a CAI/CBT production wing. We have to produce what customers want, and if that’s what they ask for... I know there is an argument to be had about helping customers see what they want but didn’t know they wanted, but that’s for another day!

While I accept that these case studies might be argued to have had effects on social equality (they claimed to - one of the reasons I selected them!), I think that the link is rather tenuous for a couple. I think they seem to be claiming that online 24/7 access equals equality - I’m sure it’s a contributory factor, but I was hoping to see a bit more than that.

I liked the fact that reading these cases studies made me reflect not just on what value the technological intervention might bring, but also issues surrounding the implementation. These are crucial for successful uptake, and key lessons highlighted in the cases include the need for substantial stakeholder involvement at as early a stage as possible, and the importance of training in ensuring that everyone understands why you are pursuing this elearning avenue.

The cases showed some nice contrasts between large and small scale, school-wide and single-instigator, big-budget and no-budget projects, and all were able to demonstrate tangible benefits.

Innovative practice with elearning - case study 4

The final case study I selected was

Swansea University Case Study: Use of podcasting in Archaeology

I picked this case study partly because I’m not sure what the fuss is about podcasting, so I wanted to see what benefits had been found. I can see it as a nice means to provide bite-size chunks of portable materials, but what else? I must be missing something.

This particular project involved the production of podcasts combining video of an archaeological site or object with an ‘expert’ commentary, made in the field. Commonly used black-and-white images in text books can be unsatisfactory, so this initiative aimed to allow the lecturer to focus on a specific issue and provide appropriate commentary.

The podcast material was prepared for use by both undergraduate and MA students. First years in particular have often not had the opportunity to visit archaeological sites and so find them hard to visualise. This project took videos while working in the field, and sometimes added commentary at the time or otherwise recorded it separately after the event. One of the aims was also to focus on student-centred learning and encourage a collaborative experience and reflective learning.

Podcasts were added to existing modules and were intended to fill identified gaps in student knowledge. While they were offered as supplementary material, they were integrated into the course’s existing VLE, allowing students to find the relevant material as they progressed. Evaluations suggested that students performed better in image recognition examinations and also appreciated the availability of digital material. Students also responded to staff podcasts by creating their own pieces and it is hoped to encourage them to share these or combine them into a resource on the VLE.

Tangible benefits reported include:

• Allowed research/fieldwork to be integrated into the undergraduate scheme, helping students understand the development of the subject and research from an early stage of study.
• Students felt they were engaged with the fieldwork - presumably this was motivational.
• Offering commentary seemed to help students engage with the sites in a more meaningful way
• Possible uses include using podcasts as a museum or site guide, using them to prepare for a real site visit, offering access to archaeological sites for students with mobility problems, to illustrate a specific point in a lecture or even as tailored support for a student who does not understand a particular issue.
• Benefits the lecturer through being required to think about how to present/discuss complex issues.
• Effective in enhancing the range of learning materials available to students.

My thoughts:

• I noted that this case study seemed to be reporting the work of predominantly one lecturer, working alone, using images for which he owned the copyright, and largely without institutional support. This was in start contrast to the extensive budgets, planning and development teams of the medical school VLE and computer-based assessment projects of the previous case studies. A nice illustration that with a little knowledge, and the use of widely available technology, subject matter experts can produce valuable elearning materials - without extensive budgets or IT support.
• Use of the materials by both undergraduate and MA students is nice example of re-use or re-purposing of e-materials.
• Integrating the podcasts into the existing VLE sounds simple, but is worth noting - supplementary materials which students can’t find or can’t relate to other aspects of the course are of reduced value, so this was a good move.
• A major drawback was in production time - about 1 hour to compile a 3 minute podcast. This was particularly onerous given that only one lecturer was producing materials. However, podcasts are clearly re-usable and ideal for sharing with colleagues and more widely, so this cost could be reduced when bespoke podcasts are not required. 
• The case study largely focused on the value the lecturer's podcasts offered to the students. However, the opportunity for students to respond with their own pieces actually offers very real potential for excellent reflective and collaborative learning opportunities which were not extensively discussed. 

Innovative practice with elearning - case study 3

The third case study from the JISC Tangible Benefits publication was

The University of Nottingham Case Study: Moving from Optical Mark Recognition (OMR) to Computer Based Assessment (CBA) for summative exams in medicine

The Nottingham Medical School used optical mark recognition (ORM) for scoring exams, and wished to move to computer-based assessment (CBA) instead. There were two key drivers for this, namely the time pressures of marking (with OMR each exam paper must be scanned, so marking time increases in line with class size) and desire to incorporate images (e.g. microscope and x-ray slides) into questions, which was difficult to do with sufficiently accurate reproduction on paper. There was also the attraction of being able to use interactive question types such as drag and drop labelling and image hotspots.

The case study appears to relate to 2003/4 when the Medical School began developing the TouchStone online assessment system and populating it with the question bank. At this time there was no institutional steer regarding computer-based assessment, and very few departments were using online assessment for high-stakes summative assessment.

The problems anticipated with moving from an OMR to a CBA approach were actually administrative, rather than technical or attitude-related. There was a difficulty finding computer labs large enough to examine the whole cohort simultaneously, and there were concerns about plagiarism that might be afforded by online exams (i.e. seeing other student’s screens and accessing forbidden materials during the exam).

This case study focused a lot on getting the electronic workflow process to support online exams right. This was clearly a large project with many stakeholders involved, including academics, administrators, subject matter experts, external examiners, disability experts and IT support. There was an awareness that by computerising all aspects of the workflow, quality could be enhanced by ensuring backups, collaborative working if any stakeholder was unavailable (within appropriate permissions-based access) and transparency of work.

Tangible benefits identified include:

• electronic access permits all stakeholders to access and work on their part of the assessment process. Also electronic storage of questions, past papers, student profiles and exam results can be analyses statistically.
• marking time was greatly reduced - has positive impacts on scheduling subsequent activities such as moderation, follow up exams, etc
• ability to include multimedia questions types, in particular image labelling and hotspots. Labelling and hotspot questions are similar to multiple choice with a diagram, but reduce cognitive load. Hotspot questions also reduce the probably of guessing correctly
• online assessment makes it simple to accommodate some disabilities such as dyslexia and other visual problems
• students can change their answer as many times as they wish before submitting - previously multiple marks on an OMR form could cause scanner errors

My thoughts:

• This appears to be a good case study of process improvement through applying technology - there have been cost and time savings as a result
• Technology also afforded better assessment - interpretation of various medical images is a key skill, but could not be adequately assessed by the previous method
• Involvement of all key stakeholders from the beginning of the process was again identified as necessary for success - this is particularly important while there are staff who are IT-averse or shy.
• The case study pointed out the inherent risks of online assessment, in particular the reliance on servers, client computers, network and power all being necessary for an exam to run. While this does appear to be adding risk to the exam process, I suspect that as daily life makes ever greater expectations of availability of these systems, prevention and mitigation of these risks becomes easier.

Innovative practice with elearning - case study 2

My next case study from the JISC Tangible Benefits publication was

Newcastle University Case Study: Use of a Virtual Learning Environment (VLE) to deliver a 'regional' medical school

This case study describes an initiative by the medical school at Newcastle to collate their learning resources online and organise them in format linked to the medical programme. There were a couple of key drivers for this, but the most notable was the pre-VLE reliance on paper-based resources and the requirement to better meet the needs of a dispersed population. In particular, students in years 3-5 of their training are not physically located at the campus, but were required to return to the medical school for a day a week in order to collect paper notices and resources. This scenario seems almost unimaginable these days, but it’s hard to tell when this project was dated. Elsewhere in the case study they refer to the predominance of dial-up Internet, broadband not yet being widely available - does this date it to somewhere pre-2001-ish? In addition to the constraints on uploading materials implied by dial-up, they also expected initial staff resistance and a slow take up given the significant change from current practice.

They had a number of fundamental philosophies in the design and integration of the VLE, including:

• data is entered once and re-used as far a possible
• the VLE should provide a user-friendly content management system, accessible to non-technical administrators
• non-technical users are empowered to independently manage their content online
• open-source software should be adopted whenever possible
• presentation and content is customised for the individual, based on their role.

The first version of the VLE was rolled out as an additional tool to existing working habits. Users were not forced into the change, but allowed to familiarise themselves at their own place, in additional to student and staff training sessions. Functionality of the VLE was gradually expanded, allowing users greater support/tools without being an overwhelming start. Within a short space of time the VLE grew to become and integral and embedded component of the degree programme.

Staff uptake was initially tricky, as expected. However, initial ‘champions’ were vital in the system’s success, promoting it. Students quickly adopted the VLE and were grateful for it, putting pressure on other staff to contribute to it. Unexpectedly there was a conceptual barrier in that some non-technical staff could not initially understand the service the LSE could provide.

Tangible benefits included:

• significant improvements in student learning, assessment, pass rates etc. Attributed to 24/7 access to VLE and scope for independent learning
• significantly improved student satisfaction with the learning process
• significantly improved staff satisfaction with elearning - and the services and feedback provided has motivated staff to improve their resources. This in turn has let to staff empowered to upload, manage and maintain their own material and to more general improvements in staff performance, widening participation etc.

The case study reports the VLE as being something of a victim of its own success, with new features expanding the horizons of users resulting in them demanding more. High usage at peak times also necessitated the purchase of a separate server. However, both of these are good problems to have!

My thoughts:

• Rather like the first case study, this initiative was about making resources available online, although with different intentions.
• The clearly articulated underpinning philosophies appeared very useful in guiding the development of the project, and combined with the stakeholder development group, ensuring the needs of various user groups were understood from the outset, allowing development of the ‘right’ tool
• Unexpected lack of conceptual understanding among staff is similar to the previous Exeter case study - underlines importance of ensuring all involved know why the development is occurring
• Students rated the VLE highly - I find it hard to imagine a university relying on paper in the way this previously did, and suspect that as schools and colleges also adopt VLEs, that students will now assume to have this type of provision as standard.
• This was another ‘putting things online’ example, but it demonstrates significant benefits, not least because of the difficult distributed working environment before the VLE initiative. There were low expectations and experience to start with, but this grew very positively in acceptance and outcomes.
• Providing content on a VLE isn’t that innovative now, but the radical changes it made to the geographically dispersed population in this study shows it was genuinely innovative at the time. Perhaps this also illustrates a good test of an innovative solution too - that it was not just a passing fad, but is now a recognised, even expected, way of doing business.

Innovative practice with elearning - case study 1

My first case study from the JISC Tangible Benefits publication was

University of Exeter: Case Study: Online economics texts

My initial thought was that “online texts” sounds like an example of the repository-of-static-texts building that we so often lament when it is argued to be elearning. (I’m not claiming that online texts are not a part of elearning tools and strategies, but that to limit oneself to that one kind of implementation is to miss the wealth and variety of practice that elearning can bring). So, I wanted to see what had been done here which was interesting, valuable, and even innovative.

This case study, although undated, appears to report an initiative from 2006/7.

The School of Business and Economics at the University of Exeter has moved towards the provision of online formative exercises for several modules. Motivations for this include:

• increased student numbers with very diverse mathematical background on some modules - raised pedagogic issues they needed to address
• a need to increase the amount of formative assessment and associated feedback
• a need to cater for international students and students from outside the School
• a need to have core modules which can be delivered by a number of people including temporary lecturers

Most students in the Business School are between the ages of 18 and 22 and enter with three high grade A levels. They are generally available to attend all lectures and classes and do not have family responsibilities. The School has already found the use of WebCT invaluable in ensuring that all students are able to access materials in advance of lectures.

However, some classes had attendance at 60-75% - lower than desired. Students were often not preparing exercises in advance of class, reducing the value of this contact time. While retaining lectures as the primary learning/teaching tool (in order to meet institutional expectations), the School introduced online exercises to be undertaken between lectures. They linked each lecture to an exercise in order to reinforce both elements. This allowed contact time to be used more productively on discussions, presentations, etc.

They anticipated a number of challenges, including:

• student expectations of weekly classes/seminars etc
• students not familiar with e-learning as not used in previous modules
• buying the set book was required in order to access the publisher’s online resources
• resources were not embedded within WebCT, where students might have expected to find them
• largest anticipated challenge was whether the students would actually do the work each week

Students and staff were trained on using the materials at the start of the modules/term. The activities were provided by the text book publishers and so were linked to the academic content of lectures. They included accessing case studies, videos and completing exercises such as multiple-choice, essay or graphical questions. Results were sent to the lecturers.

During the modules, the numbers of students logging in and completing assessments were recorded. Informal feedback from students was sought during the module and more formally afterwards. A record of log-ins was compared with exam results. There were concerns over the number of students completing the assessments each week, although numbers/percentages are not given. Students apparently did not cite unwillingness to buy the book/access code as a reason for not completing the assessments. Problems were identified in the dissemination of the approach to support staff, some of whom told students that 'There aren't any classes, just lectures', suggesting that the online classes were not proper. This issue was addressed in stuff training for the following year.

Tangible benefits were noted as including:

• pass rates/ average marks equivalent to the pre-elearning implementation of the module
• high student retention - i.e students progressing to next course
• staff reporting enthusiasm for elearning - previously they had been concerned about the cost of designing materials, and use of publishers’ resources avoided this
• e-resources mean all students on the module have access to the materials - and can access them at their own pace, revising if necessary
• savings in staff time (reduced tutorials) and space (reduced classes)
• the publishers’ materials were extensive - more than they would have had capacity to design and produce in-house

My thoughts:

• Requiring students to purchase a book in order to use e-resources doesn’t sound very progressive. I appreciate however that this is a stipulation of the publisher, and that academic publishing more widely hasn’t yet figured out how to monetise the provision of online content in the way it can books or paper journals.
• Case study states that staff will meet to reflect on the approach, but doesn’t report the outcomes of this.
• No discussion of any correlation between recorded log-ins and exam results.
• Some students said that they worked in pairs and so only one name was recorded - therefore records of participation/activity completion may be inadequate. Also, if students are penalised for not having an activity recorded in their name (the study does not suggest that they were) this would be anti-collaboration.
• This does seem to be a case of just putting activities online, although they were integrated with face-to-face teaching.
• The use of publishers’ materials doesn’t sound particularly innovative, but by sourcing pedagogical sound materials, and not attempting to re-invent the wheel, they were able to provide materials more extensive than would have been possible in house, and without putting off staff who were uncertain about the costs of implementing elearning - in fact they were convinced of its benefits
• There was a mention that there were concerns that interactive discussion opportunities wer being lost as the class was replaced with online activities. However, apart from the fact that the lack of discussion in classes was one of the reasons for moving to online activities, I wondered if the new online approach could have been modified to include discussions or interactive/collaborative activities.
• Generally this seems to be an example of a well planned and carefully supported implementation, which brings tangible benefits without the need to be revolutionary. Using publishers’ materials is not particularly ‘innovative’ (surely publishers wouldn’t be going to the trouble of producing them if no-one is paying?) but it demonstrates good practice in the successful integration of quality elearning materials and activites into existing lecture courses.

Innovative practice with elearning

In 2008, JISC produced a publication Exploring Tangible Benefits of e-learning: Does Investment Yield Interest? (2008) which looked at the impact of innovation on the learning process through the use of elearning. It includes case studies of nearly 40 examples of elearning, discussing their backgrount, context, technology used, tangible benefits and lessons learned. The case studies are available from JISC at
http://www.jiscinfonet.ac.uk/case-studies/tangible.

Week 2 Activity 2 asked us to look at 4 case studies. I decided to choose case studies that illustrated effects on learning and effects on social equality, these being two of my personal prime drivers for being interested in education, and technology-supported education. I was a little surprised that although 29 of the 34 case studies reported an effect on learning, only 20 reported an effect on social equality, and the cross-over was further reduced. However, I picked these four, with some additional influences on my selection, which I note in my posts about each study:

University of Exeter, Online economics texts
Newcastle University, Use of a VLE to deliver a 'regional' medical school
University of Nottingham, Moving from Optical Mark Recognition (OMR) to Computer Based Assessment (CBA) for summative exams in medicine
Swansea University, Use of podcasting in Archaeology

My summaries and thoughts on the case studies to follow...

Reference
JISC (2008) Exploring Tangible Benefits of e-Learning: Does Investment Yield Interest? [online], http://www.jiscinfonet.ac.uk/publications/publications/info/tangible-benefits-publication

Matrix of elearning concepts - mark 1

This is a quick post as I'm a bit behind this week. Hoping for some big catch up sessions on Thursday and Friday... trying not to panic yet!

Using the definitions of elearning concept we'd found, the task for Week 2 Activity 1 was to arrange those concepts on a matrix, showing where they might be categorised on the spectrum of Existing to New and of Formal to Informal.

This is my matrix:

A number of the concepts had me wondering about where to place them (obviously the point of the exercise!), so I also recorded some notes about why I positioned things as I did:

Blended learning - blend of new and existing, formal and informal. The fact that it includes web-based learning etc suggests it tends towards the new and informal.

Mobile learning - reliance on mobile technologies and cannot we yet assume ubiquitous access, therefore still quite 'new'. While may be used for formal access, gives greater opportunity for informal learning.

Virtual communities - initially largely informal. As there is wider uptake and integration into mainstream education, do they become more formal?

Flexible learning - I don't think this has yet been taken up by majority educators. Not yet integrated into most programmes.

Work-based learning - apprentice schemes etc have long history, therefore recognised as 'formal' training. However, OTJ aspect means training may be delivered informally.

Personalisation - widely available in informal, not-necessarily-learning-focused environments. Still a difficulty in achieving personalisation when there is a syllabus to be taught, standards to be met and need to check all material is covered, etc.

Just-in-time learning - relies on access to resources, therefore predominantly a new web-dependent method. No formal teaching - students learn what they want, when they require it. Risks lacking broader understanding as likely only to learn what is sufficient to achieve a task. May be some deep learning as result of applying it to context though.

Peer assessment - does not require new technology, but wider uptake *is* facilitated by web, etc. Not so widely used for formal (summative) assessment, and technology facilitates wider opportunities for informal assessment (e.g. commenting on blogs etc).

Collaborative learning - again doesn't require technology, but additional forms of collaboration may be facilitated by ICTs. Widely recognised as valuable medium, but still seems to be used mainly as an adjunct to more traditional approaches.

Learning objects - developed specifically for LMS-delivered computer-based learning, therefore 'new'. Standards, metadata requirements etc are a way of formalising the development. Usage is not restricted to formal context though.

E-assessment - computer supported marking available probably 20 years. Wider and more flexible uses are a newer development. I use assessment here in a summative context, therefore position towards formal axis - more informal, ad-hoc, on-demand, non-recorded or just-for-fun formative assessment is also possible though, and ICT is prime facilitator of this.

I actually found the formal-informal dimension quite hard to use, which was something of a surprise. I think I need a bit more of a think about what this really means. Also, I have a feeling that there might be some other dimensions on which it be useful to arrange these concepts. A lot of the discussion and reading in week 1 seemed to indicate innovations in elearning have particular implications for collaboration and community, so I wonder if an individual - collaborative axis might be useful to look at. This post is optimistically titled mark 1 - I'm hoping there'll be a chance for a mark 2, but don't hold your breath!