JISC

Start date: 1 March 2005

End date: 31 July 2007

Funding programme: SFC e-Learning Transformation programme

Project website: http://www.reap.ac.uk

JISC theme(s): e-Learning

Read the transformation story which highlight keys issues, lessons learned and outputs

There is a consensus in recent research on assessment in HE that current models of practice are not effective in preparing students for learning throughout life. Academic staff control most assessment processes during undergraduate study. Only in later years, if at all, are students actively involved in managing and evaluating their own learning.  Such assessment practices lead to high workloads for staff and do not develop in students the attitudes and skills required of self-regulated learners.

This project, involving departments at the University of Strathclyde, Glasgow Caledonian University and the University of Glasgow, provides a test-bed to pilot new assessment practices more appropriate to the demands of lifelong learning. Assessment is defined broadly to include both summative and formative assessment (involving feedback) and tutor, peer and self-assessment processes.  The goal of the project is to move assessment away from models where teachers transmit marks and feedback, to one where students develop, over the course of a degree, their own ability to self-assess and self-correct. Advances in information and communication technologies offer the potential to enhance learner self-regulation and to produce efficiency gains in assessment workload.  A range of technologies will be harnessed to support assessment reengineering in blended learning contexts (involving online and offline interactions).

Aims and Objectives

The aims of this project are to reengineer the processes and practices of assessment within three Higher Education Institutions in Scotland and to disseminate improved models of assessment supported by e-learning technologies across the Scottish HE sector.  Re-engineering will extend beyond assessment practices within the participating academic departments.  It will necessitate changes in processes that support assessment – in organisational structures and procedures in institutions (in registry, estates, IT and support services), in management processes (e.g. quality assurance, external examinations, course evaluation and in the ways in which student achievements are evidenced and recorded) and in individual roles and responsibilities (e.g. time spent on assessment tasks, types of support staff, collaborative work patterns).

The specific objectives of the project are that, in selected departments across three higher education institutions, academic staff will work together to:

• re-engineer assessment and feedback processes in selected disciplinary contexts based on current literature on good practice and on available e-learning tools;
• integrate new assessment practices with each other and with other teaching and learning processes in these contexts;
• develop exemplary models of formative assessment, feedback and summative assessment in relation to large first year undergraduate classes across three different cultural contexts (a post-1992, a ‘redbrick’ and an ancient university)
• investigate where costs might be reduced and the quality of assessment practices might be improved across the participating academic disciplines;
• devise workload models and performance indicators in relation to new assessment practices so as to monitor cost reductions and learning quality enhancements;
• identify the ways in which support and administrative systems might need restructuring when appropriate technologies are harnessed to improve assessment systems;
• identify the institutional policy and procedural implications of new technology-supported assessment practices;
• share the findings (processes, tools, models) within the three partner institutions and across the HE sector in Scotland and elsewhere (e.g. through the network to be established by SFC and through JISC and other bodies)

Project Methodology

The initiative will involve members of academic staff in participating departments re-examining their assessment practices in the light of current research and in the context of available e-learning systems and tools. 

At the University of Strathclyde a central project team will work with five departments (representing the University’s five faculties) to reengineer the assessment in a large first-year class.  In addition, a cross-functional task force drawn from across the support services (e.g. Centre for Academic Practice, Learning Services, VLE implementation team, IT services) will support the work of the project team and the participating departments.  Each department will be given resources to make available academic time (e.g. through staff release) to support this curriculum reengineering.  

At the Caledonian Business School (CBS) a staff release scheme will be implemented whereby ‘e-learning champions’ drawn from the six disciplinary divisions are appointed, and supported by e-learning specialists, to work with core module teaching teams to review and re-engineer assessment practices.  A project team including the coordinator and a learning technologist and the e-learning champions will support the implementation and evaluation of changes at CBS. 

The role of the University of Glasgow in this project will be to develop the software functionality and pedagogical methods of classroom communication systems (sometimes called electronic voting systems, EVS) for assessment and feedback purposes and to support the dissemination and use of these systems across the partner institutions. Most of the uses of EVS to date have been to transform lectures into occasions where more interaction and by implication more learning takes place.  In this project, EVS will also be used to transform some of what used to be done in tutorial and revision classes. At Glasgow the project coordinator, a software engineer and a member of staff from the Teaching and Learning Service will comprise the Project Team. 

The University of Glasgow will also serve as a test-bed for sharing and dissemination of new assessment practices from the partner institutions Glasgow Caledonian and Strathclyde. This will allow the robustness of systems to be tested and synergies and comparisons to be made.

Deliverables

The main deliverables of the project are case studies. 

1.      Case studies will be produced to evidence changes in assessment practices. They will be produced for the five departments within the University of Strathclyde, six divisions within the Glasgow Caledonian University Business School and one department within the University of Glasgow.  They will include:

• descriptions of changes to assessment
• their underlying rationale
• evidence of the benefits in learning quality and learning outcomes. 
• cost and workload measures showing any reduction in costs
• an account of the change management process
• an account of any changes required at the organisational, management, human resources and infrastructural processes in order to sustain more efficient models of learning and assessment supported by technology.
• report of any impact of using multiple technologies within institutions.

2.      New instruments (e.g. questionnaires measuring any of the above aspects reported on in the case studies) will be developed during the project and will be made available to other institutions.

3.      A comprehensive website will be maintained to showcase developments and the exemplars of transformational change.

4.      Staff development events (e.g. workshops) will be organised to engage staff in curriculum and assessment reengineering using advanced technologies and to involve them in the evaluation of this reengineering.

5.      Reports will be produced for JISC and the SHEFC during the development of the project.

6.      As part of the project, the University of Glasgow will develop software to improve the effectiveness of Electronic Voting Systems for assessment and feedback processes in self-paced learning situations. The specification for this software will be developed jointly by the University of Glasgow and the University of Strathclyde, and implemented by Glasgow.  While the immediate purpose is to support specific e-assessment functions during the project that were not commercially supported when the project bid was made, any code developed will be delivered as a) open source software, b) requirements for (features of) EVS software modified by experience, that may be used in future in selecting commercial EVS software, and in making suggestions about desirable features to manufacturers.

Stakeholders

The projects stakeholders include:

• the participating departments and institutions who will be supported in the reengineering of assessment practices across their institution.
• senior management and academic staff in other Scottish HEI and FECs who, as a result of the project, will be able to access models for the sustainable implementation of technology-supported assessment.
• students in Scottish HEIs and FECs, who may benefit from the adoption of better assessment and feedback processes.
• academics and teachers in post-16  institutions in Scotland, the UK and internationally, who will be interested in the case studies which will offer ideas and support for the re-engineering of assessment practices.
• the Scottish Funding Councils , as funders of the project.
• JISC who will be interested in how to integrate e-assessment into pedagogical practice. The case studies and the institutional analysis of issues will benefit JISC in supporting the sectors’ e-assessment needs.