Category Archives: Additional

Evaluating Effectiveness

Additional, , ,

During the onsite sessions on Friday 24th February, we were tasked with reviewing the UK Professional Standards Framework 2011. As a group, we mapped out our responses to the question: “What do we need to know in order to teach well?” These answers are connected to the Core Knowledge section (K1-K6) of the UKPSF 2011.

Mind map in response to Core Knowledge

Most of the ideas generated were about our interaction with students, understanding their needs and the context. There were also ideas about communication and how we scaffold information to take different learning styles into account. Upon comparing our ideas against the Core Knowledge points, K4-6 were not fully addressed. K5 in particular is of interest to me. It states: Methods for evaluating the effectiveness of teaching. While we had included a point in our mind map that said, “Making success criteria visible to ourselves and each other” we had not considered what success criteria might mean. While I used evaluation methods in my teaching, they tend to be broader topics to include a student’s entire experience on a unit or at UAL as a whole, through the CSS and NSS.

Effectiveness is a term used in the marketing industry (the subject area I teach), it even has its own awards – the Effie Awards – to identify best practice. However, there is much debate about how marketing effectiveness is measured and attributed. Renowned marketing effectiveness experts Les Binet and Peter Field wrote The Long and the Short of It, a publication that explains that effectiveness must be measured in terms of short-term marketing activation as well as long-term growth through brand building. Marketers are advised to apply the 60/40 rule, where 60% of budget should be allocated to long-term brand building and only 40% to short-term activations.

If we were to apply this concept to the higher education context, our methods of evaluating effective teaching would look quite different. Currently, our main method of evaluating the effectiveness of teaching is assessment, where students demonstrate how well they have learned. However, this is not the same as evaluating teaching as there is no guarantee that students have learned well due to good teaching; they may have prior experience of completing successful assignments or they may have read widely to understand the subject area. There is no direct method that we use to evaluate effective teaching. Evaluating effectiveness is problematic in teaching in the same way that it is problematic in marketing; what can be measured and what should be measured are not the same thing and lines of attribution are opaque.

We use unit evaluation surveys to assess how students felt about the unit as a whole. The results of these are used to improve the unit for the next time, and to identify successes and failures. In my experience, the successes and failures of a unit are already known prior to students completing the survey. Points about lack of communication or confusing lectures tend to get raised during the unit and the survey does not include questions that are specifically related to effective teaching.

It would be interesting to develop survey questions that could assess teaching effectiveness directly.

Additionally, Binet and Field’s 60/40 rule could be considered as a method for long-term success. If we were to view teaching as the 40% i.e. short-term activation, we could see that lectures, seminars and tutorials are the moments where we interact with students and capture their attention. We could evaluate the effectiveness of this portion through updated surveys as suggested above. However, there is little provision to examine long-term effectiveness as we don’t examine trends over time. One could argue that the FMP (Final Major Project) is the culmination of knowledge and that effectiveness of all the teaching over the previous three years could be evaluated at this point. However, this concept is flawed as once again it would equate effective teaching with effective learning. Another easy measure would be the amount of ‘good degrees’ that are awarded; once again a laughable concept given the way degrees are calculated and widespread grade inflation practices. We are reminded here that what can be measured does not equate to what should be measured.

A new way to evaluate effectiveness over time might be to create an extensive research methodology, which could be undertaken shortly after students graduate. Students could identify what information resonates from their whole degree programme, whose lectures and they remember and what will stick with them for the rest of their lives. A comprehensive research design would be required including qualitative and quantitative methods. The results could be linked to performance reviews and it could become a way for lecturers to stay motivated over the long-term rather than only focusing on the short-term rewards of unit completion.

Microteach: Puzzle Pieces

Additional,

The Microteach took place on the afternoon of Friday 10th February from 1:30pm-4:30pm. Three classmates plus our tutor were in attendance. The atmosphere was more relaxed than expected due to the small group and the fact that the projector screens were not working. Instead of formal presentations, we each conducted our microteach sessions at the desk, where the mood became more collaborative and fun. This reminds me of my observation session when I invited a peer and my tutor to observe a seminar session. I knew at the time that there would be little interaction from the students while I was in a more transmissive teaching mode (though there was a certain amount of information that needed to be transmitted!). This proved true as they asked many more questions when I engaged with them at small tables than when I was standing at the front of the room. While small group teaching is not always possible, there are elements of the practice that could maybe be utilised in transmissive modes as well.

I was the last to take my turn for the Microteach and participated in my three classmates’ sessions in advance. I noticed from each of their sessions that they managed to explain their concepts with relative ease by using multiple aids. Irti started with an example of how to use open source software to surface images by asking us to write an image description. This was a fun way to learn about a technology as the results were unexpected and the process was eye-opening. Nina demonstrated how to write Ekphrastic poetry by asking us to read part of a poem. This was a novel activity and – in my experience – is not done in large groups, often to ensure students with learning differences or suffer from anxiety are included. Genevieve used a multi-media approach to talk about high heels including slides about the history of fashion and a Kahoot quiz at the end. The quiz made the teaching really fun and we were motivated by the competition. I gave a short presentation about future scenario planning. Although I used a definition, I quite quickly realised I had not included a question or any interactivity in the first few slides. The activity itself was more interactive, and the simple use of paper helped to generate innovative ideas. But, I did feel that it wasn’t the best start to have chosen transmissive style teaching for the first portion, especially after participating in the other sessions.

Open Source Image Software
Ekphrastic Poetry
Genevieve Muwana and her shoes

Often – especially in lectures – I use transmissive teaching in a way that maybe de-centres the student experience. This is due to several constraints: 1. time constraints in terms of creating content; 2. the constraints of teaching up to 180 students in a room; 3. adhering to expectations of fellow staff and students. Seminars are designed to unpack the lecture content in a collaborative style, but often there seems to be a disconnect. Dall’Alba (2007) discusses the concept that lecturers must allow students to learn. This idea suggests that students take the knowledge that is transmitted as one part of the knowledge puzzle, adding to it their experiences, understanding and prior knowledge. According to Dall’Alba (2007), “Instead, [course participants] create, enact and embody the knowledges they encounter through the course to varying extents and in a range of ways, both individual and shared.”

Recently, I have been reflecting on this dynamic in the context of my own teaching. There is such a strong focus amongst the academic staff on delivering content that the pressure for the lecturer to demonstrate they are working hard, and know what they are talking about supersedes the student learning experience. As a result of this reflection, I have started to experiment with a slightly different approach. Instead of focusing on demonstrating my knowledge, I have thought more about ensuring that the students are learning by putting some of the responsibility for their learning back on them. I have been more specific about the weekly reading they should be keeping up with, and I have linked my lectures more directly with their assessment briefs so they can see where the content relates to their submission. The lectures have now become a part of the jigsaw puzzle of learning rather than the main event. I have become less concerned about proving my knowledge and more concerned with ensuring the students are involved and keeping up with what is going on, so they understand the part they play in their own learning.

Gloria Dall’Alba (2005) Improving teaching: Enhancing ways of being university teachers, Higher Education Research & Development, 24:4, 361-372, DOI: 10.1080/07294360500284771

MicroteachDownload

Streamlining the AB

Additional, , ,

At today’s PGCert onsite session, we worked on a task to a present an artefact. In my group, I presented an Assessment Brief in a standard format. Mikha presented a Miro board which demonstrated the design for a Q&A flowchart for students to assess their learning about circular supply chains. Amy presented a lesson plan document almost like a script to demonstrate how she would teach a class on how to make pdfs accessible on Moodle.

We were tasked with assessing the artefacts and thinking about how to improve them. We chose to reinvent the Assessment Brief structure taking inspiration from the Miro flow chart and the lesson plan. Reflecting on the pre-reading by Davis, 2012 it seems that Learning Outcomes, Grading Criteria and even the entire Assessment Brief require further explanation multiple times and in multiple formats across the course of a unit.

“…whilst it is important that students know what they have to do on any course of study, it is not necessarily through published learning outcomes. Learning outcomes might be seen as necessary for administrative purposes but they are not sufficient in helping students develop an idea of what they will be learning and how they will go about it. Indeed, in a highly supportive context, learning outcomes might be so generalised as to only define the landscape and the boundaries of their intended learning. The knowing of what to do becomes developmental and personalised.” (Davis, 2012)

We discussed the fact that the Assessment Brief is confusing and its format creates further confusion. Students engage with the AB as a fixed pdf document on Moodle that appears as a link. Several clicks are required to locate it and then open it. However, the AB only tells a partial story referencing the LOs, marking criteria, an overview of the submission requirements and the submission dates. It is necessary to match this information with the scheme-of-work and the unit contents to figure out which parts fit where and to know what is required to succeed. This fragmented form of communication means students lack surety in what they should be doing, and creates fear and uncertainty around the requirements to achieve a high grade.

We attempted to redesign the information into one document or flow chart that would present the information in a logical order. This is demonstrated in the images. Mikha showed a novel way to capture the scheme-of-work on Moodle using HP5 to create a kind of slideshow with a corresponding calendar. It was interesting to note that Amy as a digital learning team member had not seen this before. Mikha had arranged for a colleague to help her create it at a local level. We agreed that there is a resourcing and responsibility issue around how things are created on Moodle and by whom. There were also concerns about the further updates to Moodle this year.

Overall, it was a useful exercise to reflect on the flow of information and how it is grouped and disseminated to students. We agreed that the current process creates unnecessary confusion and could be streamlined. Amy raised useful points about accessibility that would be worth researching further.

Assessment Brief Redesign

Embracing the silence

Additional, , ,

In preparation for the Introductory Workshop on January 13th, I read some of the articles published in the Spark Journal and reflected on the concept of silence in online teaching spaces.

I chose an article called Embracing the silence: introverted learning and the online classroom written by Karen Harris, Intercultural Communications Trainer and Language Development Tutor, Language Centre published in Vol 5 / Issue 1 (2022) pp. 101–104 Spark: UAL Creative Teaching and Learning Journal.

The article discussed the nature of online teaching spaces and the question of whether silence always correlates to disengagement. Harris noted, “But here’s the thing: in a physical classroom, even if a student is silent, they are still a tangible, visible, live presence. In the online space, by contrast, the lurkers of the group can far more easily melt into the background.” As teachers, we are suspicious of silence and inactivity as we are unaware of whether the student is engaged or not. As Harris says, we are “unable to monitor them with our normal teacher senses, skills and instincts, we might panic.” I found the next sentence particularly thought-provoking. “The online classroom challenges power dynamics: tutors are no longer all-seeing and all-powerful.” (Harris, 2022)

Instead of offering suggestions to boost engagement, Harris instead concluded the article with three provocations opening up the possibility that silence in online teaching spaces is not inherently negative. The provocation I chose to reflect on is as follows: Provocation 2: Do we need to recalibrate the notion of “active participation”? When it comes to learning, is “active/passive” a false dichotomy?

On reflection, I consider that tension exists in breakout groups, between active and passive learners. The natural disposition of the learner could be reinforced, encouraging the active learners to become more active, and the passive learners to become more passive. Potentially, the active learners take the lead, completing the tasks required and giving feedback to the main room. The passive learners might prefer to use the chat box than the microphone, and might not contribute as much to the task. This can lead to frustration for both parties who might leave the group or interpret the group task as a solo effort. Conversely, both types of learners might get what they need or want from that environment. There is no real way for the teacher to be sure that the learner is getting what they want or need and therefore the online teaching environment requires greater trust. The anxiety surrounding the need to witness activity maybe speaks more to the teacher’s confidence than the student’s satisfaction. Some ways to build trust in the online classroom might be as follows:

  • Ensure there is a balance of tasks and communication styles to appeal to passive and active learners
  • Remain agile within the session or across a course of sessions to alter tasks to appeal to the dynamics of the group
  • Request regular feedback via virtual and/or anonymous means to assess which parts are working well and for which type of learners
  • Allow silence to occur and empower students to interpret the space in a way that makes them feel comfortable

Academic Practice Context

Additional

When I started the PgCert in January 2023 I was Course Leader of BA (Hons) Fashion Marketing & Consumer Behaviour, a new undergraduate course that enrolled its first cohort in September 2022. Since September 2023, I have taken on an interim role as Programme Director, Marketing & Branding within the Fashion Business School at LCF. This is a much more demanding job with responsibility for 9 courses and 25 staff members. I have been teaching a bit less since this role commenced and managing staff much more. I have worked at LCF for a number of years, beginning as a HPL, then progressing to part-time and eventually a full-time role. Prior to that I accumulated 15 years industry experience mainly in trend forecasting working at WGSN, Stylesight and Fashion Snoops as well as freelance work. My specialisms are in consumer research, fashion innovation and sustainability.