During the onsite sessions on Friday 24th February, we were tasked with reviewing the UK Professional Standards Framework 2011. As a group, we mapped out our responses to the question: “What do we need to know in order to teach well?” These answers are connected to the Core Knowledge section (K1-K6) of the UKPSF 2011.
Most of the ideas generated were about our interaction with students, understanding their needs and the context. There were also ideas about communication and how we scaffold information to take different learning styles into account. Upon comparing our ideas against the Core Knowledge points, K4-6 were not fully addressed. K5 in particular is of interest to me. It states: Methods for evaluating the effectiveness of teaching. While we had included a point in our mind map that said, “Making success criteria visible to ourselves and each other” we had not considered what success criteria might mean. While I used evaluation methods in my teaching, they tend to be broader topics to include a student’s entire experience on a unit or at UAL as a whole, through the CSS and NSS.
Effectiveness is a term used in the marketing industry (the subject area I teach), it even has its own awards – the Effie Awards – to identify best practice. However, there is much debate about how marketing effectiveness is measured and attributed. Renowned marketing effectiveness experts Les Binet and Peter Field wrote The Long and the Short of It, a publication that explains that effectiveness must be measured in terms of short-term marketing activation as well as long-term growth through brand building. Marketers are advised to apply the 60/40 rule, where 60% of budget should be allocated to long-term brand building and only 40% to short-term activations.
If we were to apply this concept to the higher education context, our methods of evaluating effective teaching would look quite different. Currently, our main method of evaluating the effectiveness of teaching is assessment, where students demonstrate how well they have learned. However, this is not the same as evaluating teaching as there is no guarantee that students have learned well due to good teaching; they may have prior experience of completing successful assignments or they may have read widely to understand the subject area. There is no direct method that we use to evaluate effective teaching. Evaluating effectiveness is problematic in teaching in the same way that it is problematic in marketing; what can be measured and what should be measured are not the same thing and lines of attribution are opaque.
We use unit evaluation surveys to assess how students felt about the unit as a whole. The results of these are used to improve the unit for the next time, and to identify successes and failures. In my experience, the successes and failures of a unit are already known prior to students completing the survey. Points about lack of communication or confusing lectures tend to get raised during the unit and the survey does not include questions that are specifically related to effective teaching.
It would be interesting to develop survey questions that could assess teaching effectiveness directly.
Additionally, Binet and Field’s 60/40 rule could be considered as a method for long-term success. If we were to view teaching as the 40% i.e. short-term activation, we could see that lectures, seminars and tutorials are the moments where we interact with students and capture their attention. We could evaluate the effectiveness of this portion through updated surveys as suggested above. However, there is little provision to examine long-term effectiveness as we don’t examine trends over time. One could argue that the FMP (Final Major Project) is the culmination of knowledge and that effectiveness of all the teaching over the previous three years could be evaluated at this point. However, this concept is flawed as once again it would equate effective teaching with effective learning. Another easy measure would be the amount of ‘good degrees’ that are awarded; once again a laughable concept given the way degrees are calculated and widespread grade inflation practices. We are reminded here that what can be measured does not equate to what should be measured.
A new way to evaluate effectiveness over time might be to create an extensive research methodology, which could be undertaken shortly after students graduate. Students could identify what information resonates from their whole degree programme, whose lectures and they remember and what will stick with them for the rest of their lives. A comprehensive research design would be required including qualitative and quantitative methods. The results could be linked to performance reviews and it could become a way for lecturers to stay motivated over the long-term rather than only focusing on the short-term rewards of unit completion.