Feedback: My Learning Edge

Hello and welcome to the Learning Experience Lab, where we're continuing to dig into the details of everything around education technology. I'm your host, Dan Hasan, and today we're joined by Glen Wheatley, course coordinator at The University of Adelaide in South Australia. Glen contacted me with a message of support back when we first started this podcast at the beginning of the year, and then, back in May, joined us at our inspirED conference, giving a presentation on the power of video storytelling in course design. So it's fair to say I'm delighted that we're still in touch and collaborating.

Now in the weeks since, at FeedbackFruits we've been taking the first steps towards doing our own course design - without giving too much away we're trying to put together a series of activities intended to familiarise and empower instructors who are moving to online and hybrid teaching. As it happened, I rewatched Glen's inspirED presentation several times as it proved so helpful to thinking about, not how to create, but how to curate the material for this course. As he rightly points out, there's a wealth of resources out there and with the right approach that can really be taken advantage of. In any case, the subject comes up in our conversation so you can hear about it in Glen's own words. And for the most part, we get into the nitty-gritty of feedback, which in a short amount of time has become my absolute favourite topic. First of all though, I asked what being a course coordinator at Adelaide involved, so let's jump right in.

‍

---

‍

[Glen] Yes, the course coordinator, it doesn't sound like a very sexy title, actually. But what it what it means is, generally speaking, it's the person that's responsible at the course level for all the material and, and any teaching staff underneath. In most cases, though, in the two courses that I teach, I only have one co-instructor for the one course. And so it's it's not, it's not the heavy lifting, but the buck stops on a course level with me. It also usually involves course design, and often the course coordinator delivers, but not always. So that's my role as a course coordinator.

[Dan] Okay, and you mentioned course design, at what level or to what extent you have your hands in the course design of, say, every course or particular domains of courses, Adelaide.

[Glen] Yes, I teach in the business school. And so it's it's in that domain, and specifically in entrepreneurship and innovation and leadership. And the funny thing is, is course design is something if you'd asked me about five or six years ago, I would have said, Yes, makes sense. But what does it mean? What does it involve? I jumped right in, I went straight from industry, consulting. Before I was in manufacturing, I went straight into course design and course delivery. So I was approached by someone I'd done guest speaking for to, to deliver a course. And I negotiated two things, namely that the content needed completes a complete overhaul. And that I wanted to redesign the course to be like, like I observed in the most engaging course I'd ever taken, namely, a course with Harvard online. And it was a completely online course, and completely engaging. And I thought to myself, that was 2015. I thought to myself, this is the future of education, if you can make an asynchronous course online, engaging, so engaging, then there's potential there. So that's when I realized that I was actually embarking on a course of course design and, and learning along the way about how to do it. But obviously, I had that as a model of how it can be done, you know, things like chunking it down, changing gears, switching over using different media, etc. So that was my very first time of ever designing a course and very first time I've ever teaching a course.

[Dan] Okay, thanks for that. I want to get into this a little bit. Now. Firstly, what made that Harvard online course so engaging? Was it the content, the delivery, or a combination of both?

[Glen] Well, the content for me was fascinating because it was on disruptive innovation, Clay Christensen. And so that that content in and of itself, I think, is quite interesting. Of course, very current, it was current then, and it's even more current now. And one of the things that made it engaging was learning from the master. So Clay Christensen himself, the creator of the theory, recorded a number of videos. Well, he was the most commonly seen face on the screen of the course. So learning from the master, there's a certain thrill, and it felt like by the end of the course, that I knew him, although I'd never met him. And the key element though, was it wasn't hours and hours of him speaking, it would be two or three minutes at a time. So very small chunks. So the critical thing about that type of course design is basically that you chunk it down into manageable bits that are easy to consume and don't lead, hopefully, to cognitive overload. And one of the things that this wasn't explicitly said, but I observed in looking closer at the course, is it was often a chunker, to have content of theory. And then a practical example. So someone from the field is actually applying that theory in the real world. And so these would be all sorts of different faces, diverse faces from different walks of life, talking about how that that piece of theory that's that Clay Christensen had just introduced, was in the real world, how they applied it, how they were doing it, and didn't realize there was a name to it, those sorts of things. And so even if it's just the idea that you're switching gears, you're switching faces, you're switching styles, there's musical elements involved, all of that leads to the sense that the hours go by and and, and it's not as the same lecture hall, the same screen, the same PowerPoint slides. It's really changing gears, I think that's that's the key trick. So that was one element, one main element, the second main element, was social learning. So within groups, there were group work involved, and there were discussion boards, and activities. So maybe that's actually a third element, the idea of activities. So polls have items for you to answer and critical with the polls is that you get to see immediately what other students are thinking, how they're responding to the same question. And that in and of itself, is valuable learning.

[Dan] Absolutely. And I think that third one is where we're going to maybe take a little bit of a focus today. But I just wanted to add, I've been getting ads on social media on YouTube of these masterclass lessons cooking with Gordon Ramsay, and jazz piano with Herbie Hancock. And I'm interested in music and cooking. And I think if I was going to take a course taught by, as you say, a master, that's got an automatic attraction to it. But maybe leadership in education, if it falls under that topic is a discussion for another time, because I mainly want to talk about that collaboration and feedback today. But one more thing about your course design experience, actually. Now, as course coordinator, Adelaide, you've mentioned how you're involved in course design, you're involved in looking at content and material, how does that actually fit into the existing curriculum and the existing didactics of the teachers? Do they have to change their teaching slightly to account for your course design innovations? Or is it something that's built to work for them, as it were?

[Glen] Well, it's because of the nature of the beast that I have had that I'm trying to manage. I do I what I design I teach, and so it can therefore be in my style. But that would be an important point if other lecturers are involved, or other teachers. But it seems to me that the maybe I'm maybe this isn't true for everyone, but I would like to believe that most teachers actually prefer the dialogue versus the monologue. And so if if that is true, then you don't need to worry about the monologue of standing up front and delivering it. Because the idea of a flipped classroom or blended learning is that the content is delivered upfront in advance, usually online, but not necessarily, then the precious time together with the students is actually in dialogue is in activities. And then it's, then you have to, of course, be strong in your material, you need to be able to be vulnerable to not know the answer to the question the student might ask, or find yourself in difficult territory, because of the emotions that are elicited based on the subject matter, or whatever it might be. But that seems to me to be much more exciting from a teaching perspective than simply spooling off a number of PowerPoint slides, like you've done so many years before.

[Dan] Yeah, it makes me think of, and I'm still doing my science communication degree at Utrecht University, actually. But we covered some things about there being a shift from the transmission model to the transaction model of communication, that instead of a top down, feeding or transmitting information, we go more towards a dialogue. I think maybe I'm just fitting it into schemas already existing in my head, but I do see this more and more often. The only thing that went against that theory of mine is when you mentioned these masters, these are experts being taught by experts. And what I've learned is that the teacher has gone from an expert to a coach. But obviously, that's not a ubiquitous example. So lots of things to think about there. And I'm pretty sure we're gonna have to have you back for a whole series on this.

[Glen] I think the coach point is really important because, in fact, I have the very easy position of coming in from outside. So I don't own any of the thinking, I don't own anything. I didn't write the textbook, I guess, is what I'm saying. And so it makes it very easy for me to to pick and choose from material from anywhere. And so actually, what ends up happening is, I have hundreds of guest speakers that teach in my course. So they may only teach for a minute or two in a single video, video clip or in several video clips. But the point is, those are the masters, those are the experts that I can bring in. And it's up to me to curate that. So it's kind of like being a museum curator that I can bring in the grandmasters and all of their work. And it's up to me to interpret and help the students through the information. And it makes it much more it's more gratifying for me to and to fit myself into being in the midst of grades, but also for the students that they see various perspectives. I think that's absolutely critical. So the coach should be doing that helping them see different perspectives.

[Dan] Yeah. Can we zoom into that a little bit more, curating different speakers and forms of content? My first reaction was, is there an upper limit to how many different speakers, different faces students are presented with? Does it become an increased cognitive load after a certain amount of new faces? Or is that only exponentially benefiting the learning experience to hear similar things from different perspectives?

[Glen] Yeah, clearly, you're limited by time if nothing else, you only have so much time in the course and with the students together. But beyond that, yeah. I mean, you don't necessarily need to see that it has the same statement made by five different people, if it's, it's basically the same. But in some situations, you might want to do that. If you were to say, let's say, Okay, well, Japanese expert says this. And isn't it interesting how the German expert in the same field says that? So from a cross cultural perspective, so in that terms, but yeah, there are limits. But in the end, it's about what points are the most important, what's in your learning outcomes, which ones you want to emphasize? And who do you need in your teaching team to help you get that point across?

[Dan] Maybe I'm a bit late to the party here. But I came across quite recently a term 360 degree feedback, which is, I think, quite a popular idea. And some of the Dutch secondary schools, especially the idea that you receive feedback, not just from your teacher and your peers, but industry experts, the principal, your parents, your friends, to build really a more holistic picture of what are different people's perspectives on one topic. So I hope that's a nice segue into the overarching theme of the topic, because today, you said, you had quite some things to say about feedback as you're learning edge. Would you mind telling us what you mean with that?

[Glen] Yeah, it's, it's, it's funny, because when I first started teaching, and being involved with course design, of course, assessments were the part and powerful part and parcel of the whole task. But it was, I don't know that I focused so much on that I focus more on the content itself, and finding and stitching together the content. So that that seemed I was very content driven. And, and I've many of the assessment pieces, or activities I got from somewhere else. So they were pre-existing. And and then over time, I started creating my own. And that's when I started realizing, hmm, am I getting the results I want? And can I, based on those results, give the feedback that the student needs. And so I stopped finding myself going back to the learning outcomes and saying that being quite strict about Okay, if these are the learning outcomes, what, how can I assess that? And then what sort of feedback is part of that? So I have decided that this year is my feedback here. It's my learning edge. For some reason why not for some reason, I was I was, I was frustrated last year, I attended, I had all these low stakes formative assessments that I'd given students and then I had to mark them off, and procrastination set in and I found myself marking them relatively late in the piece and that doesn't help any student in factor, yes, they need the mark. But the feedback, perhaps more critical than the mark itself. And so I decided that that couldn't continue. It wasn't, it wasn't the right thing for the students, I needed to be quicker with giving the feedback. And I needed it. But that also implicitly meant that I needed to spend even more time doing it, to make it quality feedback. And so it really becomes a question of how to automate, yet customize feedback.

[Dan] As you're talking, I started thinking about Cali Ellis of The Evergreen State College, she also presented it inspirEd. And she said at her college, or in her course of mainly adult learners, they don't do grades. They don't give quantitative feedback. It's purely qualitative. And yeah, I mean, from my own experience, probably yours too. We're familiar with the rush for a good grade, the the stress and the panic that goes into that in a higher education, how well not just higher education, but probably the whole journey, how important it is to, to receive that affirmation for yourself for yourself, but also, of course, for your career for your future. And it certainly doesn't seem to be going anywhere. But do you have anything else to say on qualitative versus quantitative feedback?

[Glen] Well, it is a combination, and specially I'm thinking of it as question two is a cultural question. Because I have a number of international students who come from countries where the mark is everything. And if we were to take away the marks, and they would be, they would be desperately unhappy. Yeah. And you know, whether it's going home to mum and dad, even though they're, they're grown up students, even postgraduate students, and that's the pressure that's associated with that. So, that's the quantitative and, and, and people, it's funny if people like to know where they stand, and the numbers are very number friendly as a society. But coming to the qualitative point, that's the quantitative is for the outside world, let's say, and if we say the qualitative is for the inside world, and and what one of the things that that I did with one of the assessment pieces, this this past semester, was create more work for myself initially, but I'm hoping that I can do the work up front will help me automate it later. And that's basically it was, it was a bit of a catastrophe of a project. And I say that because it was a group project, and it lasted the whole semester, people were putting groups, they didn't have a choice. And so there were some good groups, and some not so good groups. And it was 40% of the score of the course mark. And there were lots of tears. Let's put it that way. And what I did was I cut it into pieces, I met, first of all, I made an individual assignment. And I cut it into pieces, whereby people provide the, I'm helping them along the way, in the sense that they submit a project plan that's marked in quality. But more importantly, the qualitative comments are saying this is missing, that's missing, think about this, that's great, whatever. And so that qualitative feedback helps them then prepare the second step. And then the feedback on the second step helps them prepare for the third step. And so that really is the learning journey, I'm accompanying them along the learning journey. And it's been really, I don't wanna say, a surprise, but maybe there's a surprise, a pleasant upside of that is that I'm actually getting to know the students themselves individually better and their thought patterns, because I'm seeing their their work on the same topic, how they develop it over time, and how the penny drops. And you know, they go from a relatively low mark, to a really good mark, because they got it now, if they if if it had been a single submission at the end of the semester, and it would be too late to fix it, they might be able to fix it for the future, but maybe not. Or they might see that little mark and be upset and let that be it. But if I can get, you know, kill the monster while it's young, in the early phases of a project and help them on on the right path, then I feel like I'm doing more and and that they will end up with that quantitative mark they want at the end of the semester, having actually learned more during the semester.

[Dan] Best of both worlds. Glen actually wanted to look at your learning journey with feedback, right from, if you can cast your mind back to when you were growing up? How did you see feedback in your educational experience? From teachers, from peers? Did it hold any kind of value for you? I know, for me, it didn't until very recently.

[Glen] That's an interesting question. I'm not really sure anymore. I mean, it was a while ago, but I was a good student, I got good marks. And so I don't recall getting much feedback. You know, when you're little, and you get the smiley, or the star on the paper, and, and all of that, but I don't recall getting much feedback. And I, that probably was a bad thing in the sense that I didn't get used to getting much feedback. So when negative feedback did come, I, I'm thinking I probably was thinner skinned than I needed to be, because it wasn't an everyday sort of practice. And so that's, that makes me wonder how it's made me believe today, that's really important that frequent feedback is better, because then mixed among good feedback is some negative feedback, you can handle that. But if you only get negative feedback, that can be quite soul destroying. And it makes me think of my relatively recent situation, once feedback was, in fact, when I was doing my second master's degree, and there was a final project that was basically a thesis like project. And I got a good amount of feedback on the intermediary phases. But then when I submitted the final work, I only got a number score and didn't get a single comment. And I was a mature age student, as they're called here. And so I have my, my, my view of the world pretty well set. And I was angry. I was I was angry. And I said, you know, what's going on here? What, don't just give me the number, it was a good number, mind you. But let's don't just give me the number I need to know, you know, I need to learn from what was missing. So tell me what's what would have made the difference between that number, and 100%. And I didn't get that feedback. And that I thought was off criminals a bit exaggerated. But that was not a good moment with feedback, because that's what feedback is all about the growing edge of it. And so it also makes me think of another situation, the boss employee situation. You could have the micromanaging boss that prescribes everything. So it's not even feedback, it's Johnny doing this, or the laissez faire boss that almost doesn't know you exist. So it's somehow that happy middle that needs to move on.

[Dan] And this is my inference, but I think you wanted to know, where you could improve and why you didn't get 100, right? And the number does tell you, okay, it wasn't perfect. But why? And it made me think, too, when I was younger, I did Taekwondo, we will go for our gradings to get the next belt. And the score would go from zero to 100. For kicks, punches this, that and the other, I would never score more than 55, which was like, totally in the middle. And we never received any feedback on those scores. I thought I was doing great. Take it and punch it in this, that and the other. But when it came to actually knowing how to improve. I didn't know what I was doing wrong. And so that did make me very frustrated time after time after time. Hmm.

[Glen] Yes, it's, well, well, think of it, think of feedback this way. We were talking about education. But imagine you were to go home to your partner. And your partner had a completely blank face, giving you no feedback whatsoever. Or the cat or any sort if communication is old can only work with feedback. And learning and growth can only happen with feedback. And so it's so critical that it's funny because it's so automatic that we don't think about it. But in a learning situation. I think the real problem that instructors face is if you've got a large classroom or with a great number of students, and you may want to give individualized feedback to every single student for every single assignment, but it goes over your head, it becomes impractical. And so that is how we come back to the automation thing, it needs to be automated. And that that, for me also coming back to that project, the changes in that that assessment piece this semester, I've been very rigorous now, for anything that any assessment piece that I that I give students to give them a marking rubric, so that rubric gives them, it takes me a good amount of time to create the rubrics, so that the the qualitative have high value, but that it makes my life in marking much easier. It gives them a sense of Okay, if they get an ad, it means they can look for the description, the general description, and then I might provide additional comments to that. But what it also has helped me do is, is understand that there's certain patterns of marks. So depending on how well finely grained that rubric is, and how well it's built, and I can I can, I can infer that if they get the first part of the rubric, they do poorly, they will most likely do poorly for the second part. Because it because the interconnection, for instance, and so then I can see that, okay, this student is here on the learning journey for this topic for this assessment piece. So I basically need to send them back to step A, and have them get step A right before they'll ever get step B in steps B and C, right. And so the rubrics have really helped me do that. And now it has made it quicker to assess a student as to okay. When actually for the future, building up my rubric so that there's almost like a knockout point or an initial rubric section, that gives me a quick diagnosis, whether they're on the right track, before they can even go on to the further levels of the assessment piece.

[Dan] Making rubrics seems like it can be difficult. I've never tried to make one myself, but I rarely see the importance of not just the transparency to the students and yourself about what you expect and how things will be marked, but also giving structure to the course in general. Am I right in thinking that making rubrics is hard? Does it have to be, do you have advice or strategies about how to go about it?

[Glen] Well, again, it's easier for me because I come from outside and it just seems like a sensible thing to do. So I do it right. But if I were a dyed in the wool, academic, let's exaggerate this even further and say that I really prefer doing research and don't really want to do teaching, but I've got to do it. Let's make

[Dan] That makes a lot of professors, I think.

[Glen] So in that case, you know, the reaction might be my God, why are you giving you even more to do? The rubric is just, you know, just more busy work. But what it does for us, and again, as an outsider, maybe this is something that every good teacher does automatically, but I had to learn it for myself, it forces me to think about Okay, what am I really assessing here? Because it's really easy to read through something and say, Oh, that's, that's an 80%. That's a 75. That's an 85. That's a 90, whatever. It's easy to do that. But then if some if Johnny were to, or Janie would ask, okay, what's behind that 80%? Exactly what is that comprised of? And it forces you to really think about that. And that's why all of my rubrics, because I'm not teaching an English course, and I'm not teaching a referencing course, the rubric section on grammar, spelling, things of that nature. That's a very tiny part of the rubric because I'm most interested in the content that they're, you know, the theory that they learned in the course. And so I push that down way to the bottom, because for me, that's not significant. It can get in the way if it's bad. But that's so, so I and my international students are particularly happy to hear that because they often are not native speakers. And so they don't want to be scored on their English skills. They need to be able to communicate well. Of course, but it's more about them demonstrating that they understood the theory of the course and can apply it.

[Dan] and that resume,

[Glen] lyric writing is difficult. It forces you it forces the instructor to really think through what am i expecting to see, but then again, it helps later because it automates

[Dan] That is exactly there. What am I expecting to see a resonance so heavenly, heavenly as well, heavily with a conversation I had earlier on the podcast with Lillian nave talking about UDL. And I heard similar things from Linda Lee of the Watson school, that depending on what you're focusing on in assessment in the room, rubrics. That's the idea that you're giving to students. And you mentioned things like spelling and grammar. When I'm reviewing my peers' work while reading someone's thesis or something, I can't help it. I've been called a grammar nazi. And rightly so because these things just stick out for me, because I know I can give useful quotation marks feedback about that. Whereas thinking about content and argumentation? Well, firstly, I get distracted by things like grammar and spelling. That's just me. I try not to. But it's it's lower level. And I feel like I'm mentioning it all the time. But have you heard about Automated Feedback, the tool for FeedbackFruits?

[Glen] Yes, of course, it was in last week's podcast.

[Dan] Well, there you go. I won't get into it today, so I think there's plenty to say about it.

[Glen] In that the point was made, that's lower order, and it can get in the way, of course, but it is lower order. And if I were to spend time checking whether something's been referenced correctly, or making note of every single spelling error or grammar error, I would go crazy. It would double the time that I've spent, as it is I spend, I think everyone thinks I spent a lot of time doing it. I won't even mention numbers, because they're a bit scary. But it would just increase the amount of time spent on marketing and what are we looking for? If that's what I'm testing? And yes, I must, but I'm not testing. I'm not looking for that, I'm looking at the content itself.

[Dan] Yep. Couldn't agree more.

[Glen] And coming back to marking where was, marking rubrics are really the only fair way of marking not only but it's it's it's, it's very transparent. And it's very fair, because then every student will understand that they're being judged in the same matters, because the danger of the 80% is, let's say, they get off on the wrong track. And I say, oh, they're off on the wrong track. That's a 75% sort of score, and I have that in my head. But then they do something else in the paper, that's really good. So you will have rubrics where they get a low mark for some one thing, because they forgot to include it. So they get zero for that section. But they are exceptional. And other ones, I've had 100% papers that ended up getting only 75 or 80%, because they forgot a whole section. That was in the marking rubric. And so you know, and that's, that's the way it works. But if I, if I had just gone by the fact that they forgot a whole section, if I may not have marked them as, as fairly.

[Dan] Yeah, without exception. If I think about courses, which I've struggled with in my education, it's once without a rubric, sometimes not a clear rubric, sometimes not a rubric at all. And then you're writing, you're working, you're researching, but you don't know what the teacher is looking for in the first place.

[Glen] And that's so imagine, if in an academic journal, then I don't know if this is done, I'm guessing not. But if academic journals, put out rubrics as to how they assess pieces to go to be published, how how the peer assessment process functions, and they would include rubrics, that would be really fair, but but it lets it lets this basically lets the author or the writer, get into the kitchen and figure out how things are cooked. And you know, if you want to keep a black box and be the master of the games and kitchen.

[Dan] Yeah, I think that's falling out of fashion a little bit, or I hope anyway.

[Glen] But if you're a coach, you say, Okay, I'll show you how I show you how to bake the cake. Let's do it together.

[Dan] True. Okay, so I want to move on to a question that Nhi left in the chat. And that's what In your opinion, constitutes effective feedback? What are the most important elements of that, in your experience?

[Glen] Effective feedback, I guess this is making sure that it's that it's understandable, that it's that it's actionable. For the student that they can act, they can use it somehow. Because, for instance, if it's at the end of the course, and there's no more opportunity to produce assessment pieces that would demonstrate that they've that they've internalized that feedback, then then it's otherwise it's without point, but it doesn't have the same usefulness. So it's much more effective. If it can happen. Pretty close to when, when the second piece was submitted, so early on, and then it builds on other pieces. So again, if you don't get point A, then you might not get point B and C and then we all have math classes where we if we missed a day or two in or week or two of that math class, suddenly we felt like we were on another planet as far as what was being taught because it all builds on it. itself.  

[Dan] Yeah, story of my life. With maths Anyway, what problems? Have you faced with implementing feedback processes? What resistance?  

[Glen] Oh, it's mean, the only real resistance I've experienced with myself. And it's procrastination that, oh my gosh, I've got 75 papers to mark, it's gonna take forever and just get started. I've not had much resistance. And I think it's mainly because of the nature of the university situation. There's so many people teaching so many subjects and, and each student has a number of courses, that it all gets sort of lost. But it depends on the subject you teach, and how you think it's important for the future of the student. And the reason I say it that way is I teach you a corporate social responsibility course, I think that that's an important course for our society. And so I really want engagement to happen. So in that case, the feedback if helping the students in their learning journey, and ensuring that they have feedback, and that they see the relevance of the subject matter is hugely important, I think, for us as a whole. And so I'm much more willing to put more effort into the feedback process, and making sure that the students reach where they are. So perhaps coming back to these questions, it's reaching the students where they are. So they can take the next step not say, you know, become a nuclear scientist, but rather, the next step is actually to take physics.

[Dan] Gotcha. And what kind of ratio do you use between your instructor to peer feedback versus peer to peer feedback?

[Glen] I've just started using peer feedback this semester, with FeedbackFruits. And so that and but I want to, I want to increase that at this moment. It's, let's say, 10%, peer and, and 90%. Well, that's not completely true. Because of the group work. Most of my classes are very heavily group work oriented. There's a certain amount of peer feedback, but informal feedback that's going right. With feedback fruits, it's now taking a more formal nature, right. And so I want to increase that from, let's say, 10% of the formal feedback to 20, 30, 40. I don't know yet exactly how high that will go. But the point is, that the more that that's done, by peers, the more I can focus on the high level feedback, to help students make it up, make it over the final hurdle that the students help them help each other, getting over the initial hurdles, and then I help them with with a higher hurdle.

[Dan] Is that your main motivation to use more group and peer feedback?

[Glen] The main motivation is to be able to give more feedback overall. Yeah, makes sense. Okay, and more valuable. And of course, there's the learning aspect, giving feedback, again, helps people learn. And so if they didn't learn it already, when they're assessing their peers' work, then that's another opportunity for them to learn.

[Dan] So you said you've just started using these peer feedback activities this semester, right? Yeah. If you're not, not trying to be negative, have you not encountered hurdles or resistance or something with the students, because it's not always easy to get students to start along these processes if they're not used to it. I'm afraid I don't understand much about your students at what level they're at what age demographic they're at. But I mean, depending on that, it can have quite big consequences for how easy feedback is given and processed and internalized between group members and peers.

[Glen] I'm stepping into it, slowly, getting them used to it. And so since I assigned quite a bit of a low level, or low stakes, formative assessment, that's where the peer feedback comes in. It's not with the high stakes summative, the high stakes summative assessments. I'm not yet at this point, although this is the major project that I was talking about that has four steps in it. If it's a project plan, its initial results, then it's an advisory report, and then it's a video, let's say a promotional video, for the project. And that is being peer assessed as we speak. So we'll see at the end of June, how that goes. But that's what is in the whole scheme of that project, a small percentage of the whole whole project. So I think that it's important that students don't become comfortable with the idea of you featuring the feedback that they give. I think I need to do more in helping them understand how to give feedback. And the importance of it. I've done some, but I think I need to do more so that they understand that this is their future, giving and seeking feedback.

[Dan] Yeah, that's going to become an everyday essential, inescapable part of everyday life in the real world in the working world and the outside world. Participation grading. So because you've just started on this process, obviously, it would be a bit of a jump to go straight into students' marks and grade each other for summative marks. But do you see yourself working up towards, for instance, giving a portion of the overall grade down to peers grades or the fact that they've given feedback, the fact that they've been part of that process, perhaps written a self reflection?

[Glen] That does happen to a certain degree. So that's part of that's one component. So for instance, with the final promotional video that the students are submitting this semester, it's 80%. The score is given by their peers. And then there's I think, 10%, for, for giving feedback, and then another 5%, for writing comments and another 5%. For I think that's what it is for actually reviewing the feedback itself. So if there are components, we'll see how well that works.

[Dan] Yeah, because I want to say wonderful, but maybe, you know, it's not always so easy to just give that power and that responsibility away, you do have to kind of iterate and see how it works. I guess. We're finding our way forward. Yeah. So what ways have you used to get your students to see the value? And can I just ask what kind of age or level your students mostly are at or is that very varied?

[Glen] So for that course, it's undergraduate. So I used to eat that fruit in undergraduate courses. And I'll be using it for first time and a postgraduate course this this trimester,

[Dan] Okay. And as for those methods, strategies for conveying the importance or the process of giving good feedback, do you have any tried and tested strategies, any tips you'd like to share?

[Glen] I put together a month, not a whole module, but a mini module in the LMS, basically introducing feedback and the concepts behind it, and some advice and some, in fact, there's a lovely Ted Ed video about feedback that helps people understand the significance of feedback, and criticism or how to deal with critique and things of that nature. So I do prime the pump with that. But again, I think it's a matter of like any sort of new thing you integrate into a course is taking, you know, running an experiment, seeing how it goes and then iterating with it.

[Dan] Do you expect differences between how your undergrad and post grads will be processing this feedback?

[Glen] Yes, and but actually expect more differences between cross cultural differences. Because if you have a student that comes from a culture where they're undergraduates, most of our postgraduate students are international. So they come from an undergraduate situation where it was more rote learning, and right, wrong answers. And I teach things like leadership and entrepreneurship, where there are very few right answers necessarily. There's lots of gray zones. It's it's more, it's, it's less prescribed, in other words, so it's going to be more difficult, I think, helping them understand and learn how to give feedback. Because it's, if in many cultures, the teacher, the professor, is this otherwise a garden in white, like a doctor, but very highly revered? And so that's the only voice that counts in the classroom. And for me to come up and say, No, no, actually, we all got a voice here, that that can be disconcerting for someone from a very different cultural background. And so they need to learn that it's actually true. These different perspectives are in any sort of content that’s discussed.

[Dan] Yeah. I mean, this is going right back to our first episodes, as you know, that learning to value your peers, voices. I mean, maybe it's difficult for us to speak for those cultures that we're talking about. But have you found any ways of overcoming that or reassuring students that their peers' voices are still valuable sources of feedback?

[Glen] Well, some of it and you'll understand this being based in Holland yourself, where instructors are addressed with their first name, that's, that's generally the case here in Australia as well. Okay. And, and so that's, that's the first hurdle for a lot of students is actually bringing the sub bringing myself down, but basically saying, hey, look, you know, we were peers in a certain way. We're colleagues and especially a mature age student or a postgraduate student, they're a little bit older, usually. And so they've got work experience, they're not as fresh and green as they usually have, they have some ideas about things. And so it's, it's helping them value their own perspective as something unique to the mix. And particularly in a leadership course, or in an entrepreneurship course, on the Corporate Social Responsibility course, it really is more about the dialogue and seeing different perspectives than necessarily saying there's a right answer, or a wrong answer, like there might be for algebra or trigonometry.

[Dan] Yeah, in those domains, especially I can really see the value of having to be open to all these different sources, 360 degrees of feedback. I think that's a nice place to taper off. Rarely. Was there anything else you wanted to add about feedback about your experiences about education, technology in general?

[Glen] Well, I think it's, it's really understanding that Well, for me, it's been figuring out that it's my learning too that is helping me provide that is helping me learn. We generally think that feedback is about helping the other person learn. But I've flipped that and decided that giving feedback is a way for me to learn how I can do things better. And so maybe it's reassessing the assignment, going back to the learning outcomes, or simply looking at how I presented the material and then it maybe needs to be done differently. So for me, the feedback is a loop and I'm in that loop just as much as the student. Thanks very much.

--

A huge thanks to Glen once again, these conversations always leave me with plenty of fruit for thought. And if you're interested in effectively using video in your course design, or just generally interested in how it's done, be sure to check out Glen's inspirED presentation which you can find via the links in the description. And I just wanted to reiterate something that came up - communication can only happen with feedback. So whether you're a language nerd like me, or you're a social, or a talkative person, or if you work with other people, and it doesn't matter whether that's behind a screen or face to face, for the whole process, communication is happening. And why I'm so eager to keep digging into this subject is so we can bring into broader awareness, just how this process happens from moment to moment, to and fro, back and forth. How many problems arise from miscommunication? And how many problems could we solve if we took the time to try to understand it better? If you have answers to these questions then please let's talk. In the meantime, thank you listeners, for joining us on another journey of course design, communication, and of course, feedback. And remember you can stay up to date with the podcast and everything else FeedbackFruits is growing by following us on social media, visiting our website for tons of resources, and you can email me at dan@feedbackfruits.com to get in touch directly, especially if you have guest or topic suggestions - we always try to follow up on those as soon as possible. But until next time, thanks for joining us in the Learning Experience Lab.