Feedback practices in Instructional Design

Hello and good day to you. Thanks for tuning in to the learning experience lab. I'm your host Dan Hasan, and I've been talking to the heroes of education and technology for over a year to harvest, prepare, and showcase the bountiful fruits of innovation, collected from higher education faculty all over the world.

I started doing that with FeedbackFruits a year ago in written documents but now have this incredible opportunity to present in audio format, going even deeper into the experiences and insights from educators and instructional designers to share with you what I'm learning. And this episode we have quite the feast on the table, with my first recorded conversation with an instructional  designer, faculty development trainer?? And all round teaching and learning innovator, John McCormick from Brandeis University. 

Before we jump in to our chat about the foundational context and concepts in Instructional design, as well as some of the challenges of domain specific feedback, and solid course design, a quick personal note.

Last Friday I just finished writing my design thesis about augmented reality tech and informal education at Utrecht University in the Netherlands. I worked with a 5-person team to design and develop an educational product aimed at youngsters, to help them learn about proteins, making the link to human health and nutrition. We aimed to both increase familiarity and spike wider participation in this area of biomolecular science which generally hasn't been a focus at this level and age group from what we've seen. After all, invisible enzymes and bodily processes don't have the pop culture allure of dinosaurs, outer space, and other domains well known in the popular sphere. We didn't want to just design something that, from a scientists perspective, we found cool and interesting, and expect a wide target audience to share our enthusiasm. So we thought about how to make the content fun and approachable, rather than far and distant.

We did this Through the caricatured, anthropomorphised representations of proteins as miniature superheroes carrying out tasks in the body, and a carefully designed smartphone app which gamifies some of these functions in an interactive AR environment. We drew inspiration from the website amoeba sisters, whose depictions of biomolecules with faces, emotions, and personalities, we were impressed with as a way to reach that wider audience.

With our increased awareness of microbiology and human health, this project seemed like a good way to increase early exposure in this school-starter demographic to hopefully equip them with some of the knowledge needed to better understand things like macro and micronutrients, viruses and vaccines later on in life. This project was commissioned by the Epic xs, with no more guidance than "teach kids something about proteomics'', and i'm very glad to see how far we were able to get with that idea.

I'm dying to find out just where XR will be when I'm older, and I hope I contributed something towards understanding the possibilities and limitations in the field. The affective end of this, the desired outcome is a more inclusive and accessible education where different modalities are represented, different avenues are possible, and different assessments and qualifications are offered. 

As people very close to me have special ed needs, and we all could do with a good look at how we learn, I really hope to continue to see good pedagogy driving technology forward in the future. What that way is, we're still finding out. But for now, let's keep the conversation going 

Lastly, thank you to the ones who made my journey through education, helped me through 15 months of that thesis, and provided me with this current platform to speak. And now, back to the other sort of technology in education, with our convo with John.

---

[Dan] In the last couple of episodes, I talked to a few students, you know, from the Caribbean to Korea, about their experiences with different learning environments, styles, cultures, and especially how they've encountered feedback both in the context of school activities and assignments, and in a broader sense, giving and receiving feedback in various social settings, like to and from an institution or to and from their parents, friends groups. Now, having grounded some core ideas about individual feedback experiences in those episodes, I want to take it a step further and talk about both teachers' perspectives on feedback. And also from a learning design point of view, what are some of the processes, practices and theories on which feedback is incorporated into course design. So again, thanks for coming to talk to us. And maybe we can get a bit started for the listeners with some introductory bits and pieces. So firstly, a bit about yourself. What's your current position at Brandeis? Right? And how did you end up there?

[John] Thanks for having me, Dan. Glad to be here. Um, so my current position at Brandeis is Associate Director of learning design. Until recently, we were a small team within a professional school, highly technical content, like Robotic software engineering and Analytics. And we were just recently about six weeks ago, moved into the library of centralized services. And I think there'll be expanding this team, they have a few temporary right now their instructional technologists, but they'll, they might have them on our team and expand because the campus as a whole is a face to face campus, it didn't have any instructional design or really instructional technology support. Prior to that. I moved to Brandeis during COVID, I haven't actually worked on the campus yet. This is since June, previous that I worked at Lesley university, I worked in these kinds of teams from anywhere from four to 12 people in small and medium sized colleges in this region in the Boston area. So put about the past 17 years.

[Dan] Okay. And so how would you describe the field generally, as a learning design professional as a faculty development professional?

[John] The field, it's, it's interesting, it's an instructional design, learning design, and academia, I think, is a really different animal than outside of academia. There's certainly a lot of commonalities. But I noticed recently that the last two or three years I've seen some research papers talking about, you know, how instructional design and learning designs work in academia. But there's not much out there on that. And I wonder, you know, there are some significant differences. I remember recently thinking and seeing a podcast, where it was marriage him Neil and her talking about No, I'm sorry, it was a Darby Flower from the United States. Speaking on a podcast. I think it was called UDL. Some UDL podcasts. But in any case, um, she said, you know, backward design doesn't really necessarily work for faculty development, especially during remote learning, this sort of design. So this, if it's the idea well, in Europe, it's probably called constructive alignment. But I think the way that you talk about it is maybe less than lockstep than let's say, some designers might follow it, right, the idea that you're going to start with learning objectives and goals of the course, and follow that directly backwards, it just isn't. I found out my career, it doesn't really always resonate with faculty, it's kind of hard. And, in fact, even myself, and I'm designing I don't I don't, I do more of a sort of a circle around the three parts of the alignment, you know, okay. Um, so I think in the field, it's in flux, right. There have been some recent books on it. Trying to I can't think of the names offhand. But I'm talking about all the influences on instructional design, learning design, where learning design came from, which I think was your area of the world, around 2000s, and that it integrates a lot of different fields. So it's an exciting time. And it's interesting, you know, design for user experience, human computer interaction, those kinds of fields have influence on it. So there's design side, the learning science side coming from cognitive psychology, which I think is something that, you know, the learning science has been ignored. And recently, it seems to be be discussed more that the importance of it that we're really campuses, you know, higher ed needs to have more of this understanding of learning science as a stock and trade right, many more people should be many more, I have much more knowledge about that to be able to teach.

[Dan] So, would you mention instructional design and academia as a subset almost. And you'll have to forgive me because I'm a total newbie to instructional design is my first encounter with it. And this my first job. And for me, it's difficult to see how instructional design exists outside of academia or outside of pedagogy, shall we say? So what are some of the other places instructional designers use if not, in

[John] And I don't want to give a history of design. But you know, it started in the military during World War II are my understanding the right states to do things like teach millions of soldiers how to put a rifle together, right. And where else it's used, I mean, the puppy, maybe some of the most sophisticated use of instructional design is still in the military, where they're doing things like using sophisticated cognitive task analysis to figure out how fighter how good fighter pilots learn how to be good fighter pilots, because things like that is so important, or, you know, let's say airplane pilots, those things are important, because you know, life is on the line. And the same thing, maybe in surgery, and that sort of thing. Those kinds of efforts are made, but it's used everywhere. You know, it's used in organizations, nonprofit organizations, is used in corporations. and academia was probably a little bit later to the field. But I think because of that, you don't see a lot of study around how design is best done. Or you could call it instructional design, or it may be a subset of learning design in some people's minds, right. I think the move to, to learning design was, was motivated by trying to have more of a focus on the learner, right? and analyzing the learners experience, empathizing with their experience, collecting data about them, you know, doing some of that upfront analysis around the learner where instructional design was at least assumed to be focusing too much on the teacher or the instruction itself.

[Dan] Indeed, I can definitely see a parallel there with some of the things I've studied in my communication, science communication course, which was, yeah, looking at this change in trend and the progress of innovation from the teacher as an expert to the teacher as a coach. Coach, sorry, the focus on the teaching and the transmission to more of the the transaction model of communication and how the Yeah, as you said, and the learning process starts. 

[John] I think COVID has helped with that, right, to some degree, like the idea that, you know, faculty, especially at a research one institution, like the one that I work at, or thinking about, you know, if they understood how people learn, it would be easier to move them towards a more, let's say, interactive teaching kind of style that fits with the learning design, you know, view. But really, I think there's a huge gap and just understanding how people learn, understanding the learning science behind that, and then how that might guide your teaching. But I think COVID has kind of opened up that idea of focusing on the learner if only because of the empathy and the challenge that everyone's had right with remote learning.

[Dan] Yeah, I definitely see that in Some ways, and I wanted to talk about the universal elements of good learning design. But let's be more specific in academia and pedagogy. Could you talk about maybe a specific area of learning design? I think you've been working with feedback protocols and formative feedback. Yeah. A bit more about the place of that.

[John] Yeah, um, feedback is a challenge, you know, I first got involved, because when I was deeply involved when I was designing online courses at Lesley university, I just, I'm always reading the research literature. And I came to this sort of general practical strategy that I thought, wow, a lot of faculty will do this, because it doesn't add more time to their work, at least is one technique. So that, that's how I started that was probably six, seven years ago. And the technique was basically just the idea that the learner or the student needs to be in a more proactive mode. So and we do this at work, right? So if we kind of follow the good feedback practice, we might do as a professional, like, if you want help from someone on your article, if you're writing or your blog or your podcast, you might say, Hey, here's my plan, or here's a draft, what do you think about it? And you might say, here's what I'm worried about his or I'm concerned about, you might guide that right, just naturally, you say, Well, can you take a look at this as this is my main concern where I'm a little weak here. So you'd guide the feedback in that way. And then, so I that was what's been suggested in the literature that I was reading. And I was suggesting faculty that if they have, and this is a big challenge in any course face to face, or online, if you have the opportunity for students to do work on a draft: a partial piece of work, right, not a final thing that's being graded, then this could happen. So in those cases, which were pretty frequent, at Leslie, at least, I'd say, you know, if you could do this step, where you ask students what their goals are, what their concerns are, so have them be proactive, you know. And then the other two steps where the student might be, well, if there's a peer reviewing it, right, it could be different. But it could be the same if the faculty were reviewing it. So it doesn't necessary have to have a peer. But whoever's reviewing it uses a set of criteria, right, a rubric, some kind of guidance that talks about quality and shares and understanding of quality. But then finally, in this step, these this first step I mentioned earlier, and then the last step I mentioned, next doesn't take more faculty time, in fact, it probably saves them time, I kind of revealing more about the learners journey through their work, is you ask the student to write something brief on their final draft that talks about what feedback they found useful and how it improved their work. And that kind of closes the loop, right? So it makes the student it forces them to reflect on how they use the feedback on the process of feedback, which is really what feedback should be about the process, and then how they use to improve their work, right? But it also lets the instructor know, did they recognize good feedback, were they able to use it, I mean, it gives them a lot of fills in the gaps on a lot of things that happen when the instructor isn't necessarily there. And it isn't more work for them. So I used to try to sell that to every single faculty member I'd work with in an online course, which was my job. And that's how it started. And then after that I became involved with, I think it was five faculty members who already had a group who were trying to improve, they're mostly face to face at the time, but some online, student feedback, peer review processes in their courses. And we did that for five years, we went to a lot of regional conferences in New England, presented on that road, a few, maybe a paper did a lot of presentations. And so I brought that all to Brandeis. And I'm in a new place now. But I should stop talking. I have a lot to say about it. But I want to hear your questions.

[Dan] No, it's great to listen to these things. And I'm thinking about learners self regulation, about also my question, I guess, right away is, when is it too much feedback? Or when is the process too packed? Because you did mention, it's not too much more work for the for the instructors, for the instructional designers to incorporate these. But I can imagine if there's an activity with a pre reflection or prep reflection, step, a peer evaluation, a reflection and evaluation and evaluation on not just the final delivery, but also the feedback process. I think some students will find that a little bit too much and certainly some people I've talked to don't like to be too caught up in the reflection and the evaluation of the process, but prefer to, quote “do the work itself”. And that seems to be more important for them. And also considering some of the different like, yeah, the different cultural approaches to learning where some pedagogy, some tactics are focused more on memorizing material on the Yeah, remembering key knowledge and skills rather than stepping back and looking at the process looking at a collaborative or an evaluative activity.

[John] Yeah, I think I take your point. I mean, I think part of this, and I think it's the Australians who started talking about this, the reflective piece of you zoom out the furthest becomes maybe the skill, what they call evaluative judgment, right? Like the idea that students should be able to evaluate their own other's work. And the end reason for that, the most important reason is they're gonna go out in the world, and they will not have anyone to help them judge whether the work is good enough. And when I think it was David Vogt from from Australia, he's one of these researchers said, in a video, I was watching, I have this in my course, for faculty professional development, said, you know, they might only find out when they when they get fired if they don't know how to judge their own work. But that would be the ultimate reflective, like, positive. Right. But I think that thing about its peer review and feedback, right, especially peer review, is pretty complicated. I think that's one of the problems, like what you said about what's too much what's enough, there are all these different ways of designing feedback processes, right? And it really depends on so many things, including the content, the subject matter of the task. So there's not one right answer, I think that three step protocol I gave was kind of a general thing. But you know, if you work with people who teach artists, the way they do critique, which is a long standing practice in their field, and what they're doing, I guess it works, but they could benefit by learning some of the research and figuring out how to do it better. I mean, some of the social emotional challenges around the way art students get critique is pretty vicious, right? It can be pretty I mean...

[Dan] Much more personal and yeah, emotional, creative, individual experience, then, for example, writing a theoretical research paper on, you know, on an academic subject, when you make art, when you're making music, it's really like a little bit of your soul going into what you're producing. So I think that's definitely where critique can feel like a personal attack. 

[John] Exactly. Exactly. And you know, and I don't know if we can leave this on the podcast, but Leslie, the student were complaining, and maybe students are more sensitive nowadays, I don't know. That seems to be the feeling of the instructors, in some cases, because they, I know someone in the article, and they shed was this me, when the students are complaining about some of the critique practices, which are basically public review by instructors and students write their response, I saw the response of the faculty, it was basically if I can paraphrase, hey, students, you know, Buck up, this is the real world and you know, you know, it's, it's, it's a, it's a skill, so you're gonna have to learn it. But what I didn't see was a recognition that they had some work to do to teach the students that skill, right, and to do better themselves. But it is complicated. I think the gap and the research now from what I understand currently is subject specific feedback and peer review practices, right? How is it different, and I've run into some really bright and thoughtful instructors at Brandeis, even recently, in the last week, who when I bring up specifics of feedback practices in the context of the conversation we're having, they present some challenges from their content area that you know, I don't always have the answer for so there's still much to be learned. Right?

[Dan] Do you have some examples there?

[John] Well, in the area, I don't have a specific example, except in the area of things like computer science, or I'm working with a faculty member who's teaching, he's a graduate student, but he's taught this face to face quite a bit Symbolic Logic. And, you know, we get to these areas where he's, he tells me, you know, I'm not sure that there are, there is a way to do a rubric or a way to give them criteria, you know, and it's something where I'm not sure, like, so I'll often have to go and try to research a little bit understand the content much better than I do. My first step is usually is there any research showing how this is taught this particular area? So I think that's where it's missing. But there's so much and it's it's it can be complicated, and it's, it's a it's a heavy lift. I think one of the biggest problems if you don't mind me mention this is if the course is not designed to allow for formative feedback, like there is no draft work. There is no you know, project that's broken up into steps so they can get some feet, then you can't do it, right. You can't do that kind of peer review. That kind of formative feedback is just a non starter. And that is not an easy lift to make someone changed their teaching to include that. So that's where I think it's a very developmental process for teachers as well as students.

[Dan] I really think times are changing but even in my own very recent experience, I've been in courses which don't allow for specific or explicit formative exercises, for example, without going too much into detail. We had a weekly assignment in one course, where we would receive comments on how we approach text, and we'd write some questions about it. And then we had to do this for eight weeks of the course. And whichever was the lowest of the grades for the assignment weeks, then that would be like taken off, and that was the closest it got to a form of exercise where the worst thing you did wouldn't be counted. And you would receive comments, but not necessarily where you can improve or, or what could go better, just how did this work go, without looking forward without having some kind of reflection. And I mentioned to the teacher in the course evaluation, I didn't feel like I had a chance to, to learn or to grow here, apart from reading the materials, I have to mention, it was a course outside my domain, it was an elective. So I was a little bit unfamiliar with the content in the first place. But especially as an outsider student, I felt like I needed a bit more guidance and a bit more of a chance for that formative growth.

[John] You know, I think it'd be fun to talk about this. But I think there's some definite legitimate challenges that instructors have, especially with large classes, right. I mean, while it would be really helpful for them to learn even a little bit about all this research, and some of the just practical steps, and some of the tools frankly, like, you know, set of tools that you have, which can help leverage some of the scaling up, right, and some of the logistical problems you have without software. But the legitimate part is that feedback, you know, if you're talking about feedback, or peer review, right, it's a lot of work either. Giving substantive feedback on draft work throughout the courses, a lot for the faculty to leverage the peer review is a lot of work just to learn how to do it right. And also convince the students and yourself of the value of this right, and it's developmental, I think it's best done across a program, not in a single class, that way, the students are learning how to get better at it. And for each subsequent class, those instructors don't have to be as skilled, because their students know how to do it, men are in better, not not withstanding, right, there's a content connection to it, right? It's not just a skill that's devoid of content, right. But still, some of it, there is some carryover in just the knowing how the practice would be in any domain. Like the commonalities around feedback, as we said, at work, right? Like I should guide the feedback I'm getting if I can a little bit, even if it's to an instructor, like, you know, instructor meets with a student around a work of paper, let's say, this instructor might say, Well, you know, What, are you confused by what do you feel about your work, you know, trying to figure out where the student is in their head? It seems like that's pretty common. But

[Dan] Hmm. And, yeah, I mean, if I think about a degree program of X number of years, split into however many courses, it doesn't seem like, I can think of possibly any example of where there is a program wide or a degree wide, critical thinking skills, feedback moment, apart from the closest I can imagine is, sometimes you have a tutor or a mentor, who periodically will ask you, how are you doing? And how are you finding things? And without that in place, and that's definitely not always in place? Is the the, the the growth, the the trajectory of the of the learner over that course? And not sorry, over the program over the degree and not just course, by course, where you have a any number of different approaches?

[John] Yeah, and, you know, there's two examples I can think of one I know less about, but the University of Otago in New Zealand, I think, I've heard David Carless who's from the University of Hong Kong, I think, who's he's a leader in this area, Rick, you know, peer review feedback. He mentioned them as being the only program he's ever heard of with a program wide sort of peer review process. And it's around I think, being in the, like the role as a student researcher, right, moving towards becoming a researcher in the area of science. So that's the only one I've heard of. And then I know the University of Strathclyde. If you go to their website and look around for peer review and feedback, you see quite a bit and it feels like they've got a culture of feedback there. And I believe that's where David Nicol, who's another one of those names, right, was a professor for quite a while and now I think he's moved on. Maybe somebody retired. But you know, I remember you talked about school. I remember being in school. I went to school later to graduate school later, a little bit later in life. And it was probably still 20 years ago, but I remember specifically when I was in this field, right? This is instructional design and tech support. It was a good regional, large public University, University of Washington, Seattle area. And even then we weren't, we didn't have a lot of great feedback, I still remember an instructor. He was from Berkeley and is on the young side, give me this really great feedback. But it was on a final piece of work, right? I still remember opening it and saying, Wow, he's asking me all these questions. He's totally what I've done well, and why I mean, I don't see this, I didn't see it before. It's, it's too rare.

[Dan] There's this idea of that, it takes time. And also, if we're talking about those maybe more traditional teachers, the learning curve, and the resources needed to go into finding out about the research, the theories about why feedback is useful in the first place, I think, yeah, with the limited time available, teachers just want to do what they know has worked for them. And I was thinking about universal principles and how to make more time for feedback. And I think in the context of master's degrees and higher higher education, can you trust students a bit more to give each other feedback, seeing as they already have a bit of domain knowledge? Is that right?

[John] I think well, what I've seen it, it Brandeis in the Rab school where they have students who are average ages 33, and they're in the workplace, and they have some content knowledge. But also they're sort of more mature, they can get the value of the process better. I think because of their age, because they've had the work experience, probably with they've seen good and bad feedback processes themselves, even, you know, they haven't thought about it. So yeah, it's definitely a factor. But I still believe like now that we're in the library, I thought, while we could help students just be more proactive around the feedback that they are getting, or use office hours in a way that they can seek feedback better, right? Because we get better seeker feedback. I mean, all of that might change the culture over time. And even though this takes time, I think that schools, you know, universities need to have sort of communities of practice where, you know, we're not going to these formal events, to learn everything, and having to have all the theory background things can be done in a more streamlined way, right.

[Dan] I think we're gonna get better at this. So lately, and when you talk about changing the culture, I mean, where it's started unconsciously, but now it's becoming more of a palpable thing for me that I do want to see more of a feedback culture as a general thing in academia in institutional pedagogy, that it's not just about the content, the skills, the performance, the constructive alignment, the outcomes. It's about the process, the growth, and yeah, your self-reflection. I've seen this in particular with a university in the Netherlands, who have something that they're pioneering it seems called Design Based Education, every step of the way in group projects. And it doesn't seem like too much like we talked about earlier. It's not overburdening. But there are so many more ways in which feedback between students, students self reflecting the teacher to the student, where that feedback is being processed. And it might not be a feedback, feedback, specifically on work or skills again, but yeah, on the general process, what do you think you need to do next? How's it going?

[John] Yeah. And I think when we talk about reflection, I know sometimes reflective exercise feel like hoops to jump through for students. But I think we're not where I'm talking about, we're talking about reflection. Now we're talking about some of the processing, like, for example, even when that instructor 20 years ago, gave me questions and feedback. And I didn't have an update to do I didn't have another draft, I still thought about the questions and what the answers would be. I was really happy. And it was, it was just like, so rare to see that that I still remember it right.

[Dan] Yeah, it would really be nice to see a community of practice emerging where yeah, feedback becomes more of a mainstream more of a central component, rather than something that's latched on to some course.

[John] Yeah. And I became interested in because it just seems so central to learning, right? I mean, we get to some kind of confused point or some cognitive dissonance, that's kind of the center of an opening to learn. And then we get some feedback about our performance or our thinking. And to me without feedback on this formative work, I don't know that I would call it teaching, frankly speaking, you know. So whatever the challenge is, we need to take it on, you know?

[Dan] I've got, I've been noting down some of the names, you've told me and I think I've got a heap of research today.

[John] Oh, yeah, no, sorry. I mean, I've been, I've been thinking about this for five or six years. And I get, you know, sometimes I go down a rabbit hole, maybe it's the inclination of people who get into the field, but sort of like once you know enough about something you sort of want to know as much as you can and keep up with it. Because I've really seen the impact of understanding this on some of the work I've done just meeting with faculty being able to, you know, bring things in as can be done practically speaking for them, doing professional development, you know, I have a whole week and a full week online course, that's just about feedback. And it doesn't just just focus that week, but we do some real practice that week. And then following that in the other weeks, around different content areas, we continue to do the feedback around that content, that give each other feedback. So they practice what they learned in that week. So I probably wouldn't have done 25% of the course on feedback. Five years ago, you know, if I created that course.

[Dan] Absolutely. Yeah, I'm thinking more about now, how would you call it cost specific domain specific feedback, rubrics, protocols, processes can be applied? Because obviously at Feedback Fruits, we're trying to work with instructors from every imaginable domain in higher education, about, you know, what is the optimal feedback? protocol for your course? Are you talking about self reflection to peer review, group review? And there is no one catch all there is no one size fits all. 

[John] Right. Right. And, yes, we're seeing about the different areas. So this is trying to there's some research out there by some people that are looking at this, but I lost my train of thought, sorry. No worries. Um, yeah, I think it takes it takes more than one course. And if it does work in a course, it has to be something that's the right grain size, I guess, and fits that faculty, or they have to be really good at. Because I've seen faculty work on this for years. And that's why Initially, I saw faculty getting together because they were doing it, because they were in fields like drama, where it was used anyway, or they were in fields with writing was important, or education. And they so they knew was important, but they weren't being successful on their own. So they got together, right, and I happened to be the person who was already thinking about it. And I was the person doing all the research. So I would come with, like, here's what the recent research says, here's how we can, you know, integrate it into what you're doing practically in your different courses. So I just and they were people who really thought a lot about this and wanting to get better. And they were challenged, right to do it. So I don't think it's an easy lift, to move forward on your own like that. And so many instructors, they're isolated, and when it comes to teaching, and sometimes they're embarrassed to talk about that they aren't, you know, they could get better at what they do, right.

[Dan] The path of learning is one full of mistakes and acknowledging those areas of weakness. I'm way too new in this world to be talking about how trends and areas have changed in terms of societal and cultural norms. But I do feel that at least where I'm living, I don't want to generalize, but it becomes more normal to talk about yourself to talk about your feelings to talk about your personal journeys and growth and development. Yeah, and I think that's also about feedback, isn't it? feedback to oneself, growth as a process that you can share with people?‍

[John] Yeah, yeah, for sure. Um, you know, what I've noticed over the last, I don't know, two to five years is this research on feedback and peer review, it's just seems to be exploding. It's just really growing quickly. And it's nice. It's nice to see. Like, there seems to be a community that's international now, where I know this is a researcher named Naomi Windstone, from the UK, working with David Carless from Hong Kong, and they're starting to work with the people from Australia who have been, you know, a bunch of leaders there. And it's nice to see it coming together and growing. And I'm hoping it's going to have an increased impact, you know, in higher ed.

[Dan] Yeah, certainly. Again, I don't know what the contents of teacher training and faculty development courses are. But I would like to imagine that feedback is now going to take an even bigger role as something that everyone everywhere can do. Not necessarily should do in every possible course domain setting, but that is a fundamental of pedagogy, not just this transmitting this unilateral flow of knowledge from one to the other, but that is a transaction and that we all need to be aware conscious of where we are, where we're going, what's going wrong, and what's going right‍

[John] Yeah, and while there's something else we're talking about, people like, I think it's David Nickel, and some others in Australia, they're talking about other ways to get feedback that might not involve, you know, this sort of ideal dialogic sort of structure, right, or getting feedback from individual things like looking at exemplars right that have been carefully chosen pieces of work, maybe pieces of work that have been sort of, they work in a sort of graded structure where the they get more complicated. So for example, and maybe this could even work with a set of tools like yourself, but I've seen others, where if you set, let's say, a self paced series of exemplars where students can practice, you know, evaluating that work, and getting feedback about their evaluation, they can almost be trained in a self paced individual way. That's not the answer to everything, right. But it's something I haven't seen being talked about very much where, you know, you just need to compare your work and your thinking to other work. That's the essential that you actually don't need a human being. You could use learning analytics and artificial intelligence. It's not, it's not ideal, but I'm just saying it's a possibility right now.

[Dan] Now, while we seem to be moving more towards AI, and education, these will definitely be topics in future episodes. So yeah.

‍[John] We need all the help we can get right. So it's not like we shouldn't use all the tools. And we shouldn't just throw people into these systems, we can also have dialogue and peer review. And, you know, all of it right?

[Dan] Yeah, I think the thing would be that all areas have an activity of a learning activity, which can be automated, which can be assisted by AI, the basic, the structural components, Let's aim for that. And then let's leave the higher order functions, yes, executive processes for the teacher, those things that need the human touch, and the ...‍

[John] Social emotional pieces, right. Because I know a big part of the protocol that we created at Lesley university with the instructors was setting the environment, right, making it a comfortable place to be able to be open and trustful. And if you don't have that, the rest of the process is not gonna work peer review. And that's a lot of work on that.

[Dan] That makes the other person but you're operating in an environment as a safe space, secure learning environment where you know that the feedback, excuse me, the feedback that you're giving and receiving isn't meant to be hurtful. It's not meant to be a character assassination, but it's about the work and it's aiming towards improving. And again, I think that's something that maybe especially younger students are a little bit afraid of, or unsure about, especially when it's coming from their peers and not from the teacher. Because, however, we want to talk about the teacher, as a coach or expert, there is still instilled in us, the idea that the teacher is the person who knows best, their feedback is the most valuable. And so you can count on them, as opposed to if you're receiving feedback from a peer the same age as you, you know, fresh out of high school, 80 and 20. What do they know about the subject? Who are they to tell you how that is?‍

[John] That is right. That's one of the challenges of whether they know enough about the content to actually give feedback. And then this is the skill of how the language around giving feedback well, right in that sort of way that doesn't sound hurtful, but also is critical, right? And gets at the points you want to make. And I've been in someone's class, actually a physical class, which was a sort of a screenwriting class right? Now the students already have, I think, through other classes, that idea that critiques are important, but I was kind of blown away at how good they were at giving feedback verbally in a class and a whole class environment. And so I people can get really good at this. But at first, it's not gonna be great, the dialogic piece and that's why I think sometimes the practice alone or the anonymous peer review within software can be a good first step to learn the process and some of the language first, you know, without having that social emotional layer that is stressful.

‍

---

‍

John, it was a pleasure to talk to you and you really opened my eyes to the wider field of instructional design. I started out thinking that this was something done just in schools and the like, but seeing now that its use is ubiquitous, raises the questions of domain specific learning and development processes. What are the overlaps between how a soldier and how a student learns? Between a business school teacher and a corporate account manager? Learning doesn't start or stop at school, it's happening in every institution and outside of them too. Thinking about the learning experience at every different level and location is really giving me good brain food to ponder for the coming weeks.

And funnily enough, the UDL podcast John mentioned at the start of our chat turned out to be another fruitful avenue for research. I actually managed to get a hold of Lilian Nave, host of the ThinkUDL podcast and we've already had a great session talking about that framework, as well as some of the quirks and changing environments of remote learning and digital communication. I can't wait to share that with you next time, so make sure you don't miss our next episode.

It's been a huge privilege to spend time with these great minds and who knows, with the exponential acceleration of tech, innovation, and communication possibilities, just where we'll be, 2, 5, 10 years down the line. Please join me in imagining a utopia for an inclusive, accessible paradigm of education and instructional design, and let's be transparent about the problems and the difficulties we face now so we can be even more sure about the solutions we want to see.

As always, your listener-ship is appreciated and your feedback wholly welcome, so don't hesitate to get in touch with your comments, queries or questions. And remember, you can find more on these topics on the FeedbackFruits website, by following us on LinkedIn, twitter, Facebook and Instagram. Or send me a personal message at dan@feedbackfruits.com. you can now catch this podcast on stitcher, as well as all the other platforms you know and love, especially SoundCloud where we started up and continue to release on first. Hope you had a fruitful time listening to the learning experience lab - See you again next time!