Here’s an evaluation to conclude my documentation of the flipped classroom experiment. There were four main sources of data for assessing the strengths and weaknesses of the course and our application of the flipped classroom model: (1) reflections by the teaching staff, (2) attendance records taken in class using attendance software and engagement monitoring via the VLE (virtual learning environment), (3) results of assessed student work, and (4) formal feedback acquired online via the University course monitoring and feedback system. So as not to pre-empt our publication of an academic article on the experiment I’ll just summarise the results here.
The lecturers’ perspective
I’ve already suggested that the flipped classroom format enabled the teaching staff to cope with a large student group. Initial ad hoc comments from students about the lectures were very encouraging, which bolstered the lecturer’s enthusiasm for creating further content. Some of the overseas students seemed to enjoy the travel dimension of the lectures, as several lectures happened to be shot in interesting places in British towns, museums, the countryside or on other campuses.
Throughout the semester the teaching staff noted how much more engaged the students seemed to be than in previous years. Attendance was high, there was certainly a lot of group discussion.
The interviews and reflective reports (assessed submissions) were intriguing to watch and read, and indicated a level of sophistication we had not observed before.
The interview idea skews topics towards popular themes such as Internet addiction, social isolation, emotional machines, if machines will take over the world, and other high profile mass media themes — rather than how digital design is affected by metaphors, Marshall McLuhan’s theories about cultural transformation, and the play element in creativity.
So perhaps the interview format skewed the student’s interest towards themes that appear in the mainstream media.
Attendance and engagement
It is possible to monitor page views via the VLE and video views via Vimeo’s statistics page. The numbers of video views tapered to about a half towards the end of the 11 week course, as did the numbers attending the class. That’s not usual in any course, but disappointing as we would have hoped for near 100% engagement in the lecture content at least.
Student performance
We had the marks for the assessed submissions, a class quiz conducted in the last session, and the attendance record. Using simple analysis on an Excel spread sheet, there appeared to be no correlation between attendance, page views, the quiz and final performance. A few students with poor attendance performed very well, and some students who attended fully performed poorly. Hopefully we cannot infer therefore that the lectures and classes were unnecessary. But we think it likely that the assessment regime was not tuned as well as it could be to the lecture and class content, an observation confirmed by some of the student feedback.
Feedback from students
Feedback from students features prominently in any course assessment. The response rate to the centrally administered course evaluation questionnaire was 70%. The approval ratings for the various aspects of the course were at the high end of the scale, and overall at around the 4 out of 5 mark. The evaluator “Overall I am satisfied with the quality of the course” averaged 4 with a wide spread, and “I would recommend this course to other students” averaged a little lower and with an even wider spread.
We feel that the spread for both these responses indicates divided opinion on the merits of the flipped classroom approach and our implementation of it, rather than the course content. Amongst the positive comments we read: “The model of the course is original to me, and the presentation part really made me more confident to express myself. Video made by teachers provided me with useful knowledge, I really appreciate it.” “I really like that the lectures are available online so we can go back and re-watch them. Some of the class activities, like recreating scenes from Metropolis or asking online oracles, were fun.”
“This course is delivered in various forms and features well-designed, experimental class activities: making films, interviews, counting beans, … as a result of which we have acquired critical thinking and analytical skills. There are chances of working in groups and giving presentations.”
Criticisms
The section of the questionnaire labelled “Areas for improvement” yielded some critical comments about the approach: “I found it odd that we had to watch/ do about 1/2 hours worth of work before coming to the lectures but then didn’t really do much when we got there apart from some activities that were sometimes useful but also sometimes not.” “The weekly “discussion” sessions were just not working for me. Go back to traditional lecture (but in a small group, 80 students is not MSc size) or seminar where a particular paper is discussed in detail.”
One set of comments pertained not just to the flipped model, but its implementation: “The ‘inverted classroom’ type set-up was not used effectively at all. In-class activities often bore little relation to video lecture content.” This respondent also highlighted language issues in the class. Someone felt that the class period was just a time-filler: “It felt like the course was just there to keep us busy.” It’s interesting that none of the negative comments related to the recorded lecture content, but the relationship between the lectures and the class activities.
Lessons learned
Here I will just focus just on the “flipped” aspect of this particular course. Bergmann and Sams advise that having dispensed with the need to deliver a lecture in front of the class, the teacher can circulate within the classroom and answer questions from struggling students. This approach is perhaps applicable in the case of their domain of maths and chemistry, but our discursive subject matter invited other approaches.
The video content was not in question, but as we predicted the class activity provided the greatest challenge. In this case we were uncompromising in delivering the flipped classroom model. The two hour class sessions contained no lecture material. This was not a hybrid implementation. In any case, the particular space did not really invite attention on the lecturer as the students were arranged in groups sitting at tables. But we have lectured under similar conditions before, and the spatial arrangement can work adequately, especially if the lecturer uses a roving microphone. Drawing on a dramaturgical metaphor, the lecturer in that setting becomes a cabaret performer, an after dinner speaker, or a roving magician.
Observers have examined the critical relationship between recorded and live performance in the music business. Audiences attend live concerts to see and hear entertainers perform the same material they could experience through YouTube videos. Prominent academic speakers also attract live audiences, even though people can read their books or watch them on TED videos, even if the live lecture delivers the same content. By several accounts distributed recordings increase the appetite for live concerts and public talks, which in turn becomes the main revenue stream for many musicians and celebrity speakers.
What live lectures deliver
Translating this emergent relationship between the recorded and the live to the more routine world of the university lecture: live lectures delivers something that videos cannot. Apart from content, going live at its best amplifies the potential for: emotional engagement within an audience, empathy between speaker and audience, and setting the mood such that audiences are receptive to content. Gathered audiences cultivate and share moods and contribute to a receptive “frame of mind.” Distributed audiences and co-learners operating online can achieve this, but bringing people together provides immediate access to moods conducive to learning.
In light of that proposition, to repeat or extend the video lecture content in the classroom may not be appropriate or efficient, but an improvised twenty minute summary of the pre-recorded lecture content or its highlights might be, including responding to questions from students confident to speak out within a large class. In our implementation of the flipped classroom I feel we missed out on this opportunity to develop rapport and enhance learning by capitalising on the lecturer’s presence in the class, and associating the lecturer’s presence more strongly with the lecture content.
It seems the students did want to hear from the lecturer “live,” rather than see the lecturer as an organiser of activities. Inevitably we will introduce other revisions into the course next year, including a closer coupling between work submitted for assessment and the course content. But we feel that it is worth persevering with the flipped classroom format, by this or any other name, as university level learning and teaching continues to be influenced by the development of distributed digital media.
Bibliography
- Bergmann, J. and A. Sams, Flip Your Classroom: Reach Every Student in Every Class Every Day, Eugene, OR: International Society for Technology in Education, 2012.
- Caputo, J.D., Radical Hermeneutics: Repetition, Deconstruction, and the Hermeneutical Project, Bloomington, In.: Indiana University Press, 1987.
- Cohen, S., Folk Devils and Moral Panics, St Albans: Paladin, 1973.
- Coyne, R., Mood and Mobility: Navigating the Emotional Spaces of Digital Social Networks, Cambridge, MA: MIT Press, 2016.
- Derrida, J., ‘The principle of reason: The university in the eyes of its pupils’, Diacritics, 13, 1983, 3-20.
- Holt, F., ‘The economy of live music in the digital age’, Journal of Communication Inquiry, 37, 2013, 113-127.
- Krznaric, R., Empathy: Why It Matters, And How To Get It, Croydon, England: Ebury Publishing, 2014.
- Kurzweil, R., The Singularity is Near: When Humans Transcend Biology, New York: Penguin, 2005.
- Ofcom, Report: Social Networking: A quantitative and qualitative research report into attitudes, behaviours and use, London: Ofcom (Office of Communications), 2008.
- Ray, T., ‘The ‘story’ of digital excess in revolutions of the Arab Spring’, Journal of Media Practice, 12: 2, 2011, 189-196.
- Saville, B.K., A. Gisbert, J. Kopp and C. Telesco, ‘Internet addiction and delay discounting in college students’, The Psychological Record, 60, 2010, 273-286.
- Shaffer, K., ‘Homework as a social justice issue’, Digital Pedagogy Lab, http://www.digitalpedagogylab.com/hybridped/homework-is-a-social-justice-issue/, 2015,
- Snodgrass, A. and R. Coyne, Interpretation in Architecture: Design as a Way of Thinking, London: Routledge, 2006.
- Turkle, S., Alone Together: Why We Expect More from Technology and Less from Each other, New York: Basic Books, 2011.
- Ulmer, G.L., ‘Textshop for post(e)pedagogy’, in G.D. Atkins and M.L. Johnson (eds), Writing and Reading Differently: Deconstruction and the Teaching of Composition and Literature, Lawrence, Kansas: University Press of Kansas, 1985, 38-64.
Notes
- The images above are from a class video where students re-enacted scenes from Metropolis.
- Also see companion posts: Flipped Classroom 101, Flipped Classroom 102, and Flipped Classroom 103.
- [note added 1 June 2017] For full discussion see our journal article
- Coyne, Richard, John Lee, and Denitsa Petrova. 2017. Re-visiting the flipped classroom in a design context. Journal of Learning Design, (10) 2, 1-13. https://www.jld.edu.au/article/view/281
Reflections on Technology, Media & Culture 