Author: Liz Masterman (Oxford University Computing Services)
Publication date: 9 July 2008
Publication Type(s): Reports
JISC theme(s): e-Learning
Programmes: e-Learning Pedagogy programme
Projects: Constructing 2 Learn
Computer-based modelling is playing an increasingly important role in scientific topics as diverse as ecology, epidemiology, economics, sociology, animal behaviour, and climate. There is a growing awareness that university students in these fields need to learn how models are built and how they work, and yet they do not typically have computer programming skills. The Constructing 2 Learn (C2L) project aimed to remedy this by developing a tool that enables students to build models by composing high-level pre-built components, by integrating the tool with a learning activity management system, and by creating and trialling a few computer model construction learning designs.
The goal of the project was to support teachers and students in the building of models in the social and life sciences. We developed a modelling tool called the BehaviourComposer that students without prior computer programming experience could use to build, execute, and analyse serious models in their subject of study. We integrated the tool with LAMS 2.0 and built a few LAMS sequences and two activity designs that were used in our evaluation sessions. so that learning designs that included use of the BehaviourComposer could be authored, executed, and monitored within that environment.
Pedagogic value of the BehaviourComposer
The underlying aim of the C2L project was to build a tool that enabled students to create models of complex phenomena in the social and life sciences with minimal programming knowledge, and this is reflected in the motivation of the 'Sugarscape' tutor in particular.
However, it is difficult to assess the extent to which this aim was successful, given that
- students’ interactions with the BehaviourComposer were heavily directed (through the worksheets) so they did not have opportunities to experiment freely with it
- the activities were not formally assessed. Moreover, their own feedback is mixed: some found working with the tool relatively straightforward despite their lack of previous programming experience, while others who had done programming before felt that they would not have progressed without that existing knowledge
However, although the learning outcomes may have been in doubt, the 'Infectious Disease' sessions indicated important implications vis-à-vis students’ meta-theoretical understanding, which can be considerably affected both by the properties of the modelling tool itself and/or the way in which the ontology of the world being represented is implemented in the model. The choice of R – a freely available tool – over a dedicated tool for modelling differential equations led to a blurring of students’ understanding of the stochastic element in the spread of infectious diseases, while the inclusion of a constant contact rate in the model supported by the BehaviourComposer clouded their grasp of the effects of different forms of social networks on that spread. An important “learning design” issue, then, is to furnish activities that facilitate students’ understanding of the phenomena being modelled and, hence, enable them to make predictions and solve problems regarding those phenomena, while protecting the students from complex, but essential, meta-theoretical issues until they are ready to engage with them.
Challenges to evaluating innovative technologies in an HE environment
Our initial research questions were ambitious and relied upon extended use by interested parties. However, although Ken Kahn identified, and met with, a number of interested academics, we were obliged to curtail those ambitions in view of the limited penetration of modelling at the undergraduate level, the early loss of key supporters of the project and the concentrated nature of the university curriculum which makes it difficult, if not impossible, to replace pre-specified components of the course with experimental sessions of this kind – problems that are almost certainly not unique to Oxford University. Inserting optional sessions did not really work in the 2007 'Sugarscape' evaluations, particularly where the evaluation session preceded the class in which the students were to learn about the Sugarscape model in detail.
Greater success, in terms of meaningful data, was obtained from the 'Infectious Diseases' sessions, where the sessions were more fully integrated into the curriculum. The shortcoming here lay in the non-assessed nature of the task, which meant that learning outcomes could not be measured in an objective way. However, though the pre- and post-session interviews with the tutor yielded valuable insights into students’ misconceptions and the challenges associated with representing the world in cut-down form (as discussed above).
Contribution of the C2L evaluation to our shared understanding of D4L
The community’s understanding of Design for Learning (D4L) is continually evolving, from early definitions such as Dalziel (2003) and JISC (2004) to a recent conception that foregrounds the 'design-like' skills needed by teachers in planning for e-learning: for example, the representation of ideas in a more explicit way, the re-interpretation of the curriculum in new contexts, and continuous evaluation (Beetham, 2008).
C2L was rooted in Ken Kahn’s longstanding interest in learning through modelling, programming and computer animation. However, within the D4L programme it was conceived primarily as a technical development project (demonstrated also in the embedding of the BehaviourComposer as a tool within LAMS) and lacked the emphasis laid on the pedagogic aspects of learning design by projects such as DeSILA and eLIDA CAMEL, with which the author of this report was also involved. This is not a criticism of C2L – which indeed accomplished much – but it explains why the project made a smaller contribution than others to our collective understanding of D4L. True, the BehaviourComposer made it possible for both tutors to enhance their students’ learning experience, but this does not require an explicit D4L framework to achieve, and there is certainly no evidence of an alteration in perspective in the post-session reflections of the second tutor. Both he and the first tutor implemented the BehaviourComposer within the context of their existing pedagogies.
Future development of the BehaviourComposer
Since the official end of the C2L project in December 2007 the BehaviourComposer has been used again for modelling Sugarscape (covered in this report). A successor project funded by Eduserv, Modelling4All has been under way since August 2007. The new project has a less pedagogical focus, in that it aims to enable and support a community of model-builders comprising both students and seasoned researchers. Both the software architecture and the interface of the BehaviourComposer have been extensively revised, and a number of usability issues raised during the C2L project have been addressed.
From an evaluation perspective, the Modelling4All project poses different challenges (how can one defined and evaluate the 'success' of such a community?) from evaluating a modelling tool within a formal pedagogic context. Yet the evaluation of the BehaviourComposer has shown that much work still needs to be done to design and carry out effective evaluations of modelling tools within formal learning contexts in order to yield the kinds of findings (and findings that are not just centred on the measurement of learning outcomes) that will persuade educational institutions to appreciate the pedagogic value of modelling and implement it as a teaching and learning technique on a broader front – and, hence, to furnish online communities such as Modelling4All with the members on which they will rely to thrive and demonstrate their own value.