Learning and teaching practitioners face three challenges with feedback: improving the quality of feedback, improving its uptake, and measuring this uptake. Edword, an online tool, helps practitioners overcome all three. It has been tested in an ETH proof of concept and appears to be adaptable to the demands of diverse disciplines and a variety of learning and teaching contexts.
Improving and Measuring Feedback and Uptake
Here, we explain how Edword, an online tool, improves feedback, improves uptake, and measures uptake. Our experience of this tool was gained in a pilot project in FS20 involving four writing instructors and 167 course participants at the Language Center of UZH and ETH. The Language Center has since decided to continue using Edword, and a survey of users also indicates that many other disciplines may benefit from deploying this flexible tool.
Improving Feedback
Effective feedback shares some characteristics across practically all disciplines: it is consistent, for example, in identifying similar problems for different students and cohorts. At the same time, it is constructive; it both identifies problems and advises students how to address these problems in language adapted to the students’ stages of learning.Edword enables L&T practitioners to develop comments that provide complete explanations of both problem and solution and then reuse these throughout and across courses. They can also be transferred and adapted as necessary for use with other cohorts, and they can be shared between L&T practitioners. Comments can incorporate links to online resources, images, audio recordings, and exercises, so they are adaptable to a broad range of student groups and levels. Our 167 course participants were asked to provide their own feedback on their experience of Edword; 87% of respondents reported a preference for receiving feedback through Edword over all previously experienced methods.
Of course, preparing a complete set of comments may represent an investment of several hours for even a single course. However, once prepared, these comment sets can be reused many times; each comment can provide far more detail than is available through more traditional methods, and a comment can be applied in a second or two.
Improving Uptake
The uptake of effective feedback is critical to successful learning. Accordingly, feedback should be delivered in the most immediate way possible, to minimize the effort of effective uptake, and it should be prioritized so that students encounter the comments most important to their learning first rather than the comments that happen to have been given first.
L&T practitioners are able to prepare comments, as mentioned above, so that solutions appear directly beside students’ assignments; thus, students relate the complete feedback comment immediately to their own work with minimal effort. Moreover, L&T practitioners can prioritize their comments in Edword before releasing assignments to students.
Measuring Uptake
Measuring uptake is as critical as improving feedback and uptake because it can confirm the effectiveness of these two and help guide further improvements. Prior to using Edword, we as writing instructors had sparse information on feedback.
In contrast, Edword provides two measures of uptake, both of which are taken while uptake is undergoing. The first is a simple measure of students’ reactions to feedback. On completing uptake of each comment, students must rate the comment as helpful, neutral, or unhelpful. Although this scale is only coarse, responses to it are collected for every single comment. The second measure is objective: the time each student spends with each comment is timed. Both these measures are collated automatically in Edword, and the fine-grained detail this process provides can be used to improve individual comments, comment sets, and broader patterns of feedback practice.
Conclusion and Outlook
All of the advantages outlined above certainly apply when providing feedback on students’ scientific writing. Several aspects of Edword and its usage indicate that it is suitable for a range of learning and teaching contexts. Assignments can be uploaded to Edword directly from both Word and pdf documents, and it reproduces mathematical notation and equations, tables, and complex figures without difficulty or any extra effort, so disciplines using these can easily adapt Edword to their requirements. In our survey, we asked Edword-using students to suggest disciplines in which they envisaged Edword being useful. Their responses included fields as diverse as marketing, economics, biology, mathematics, and “any kind of course where a written assignment has to be handed in.”
- Questioner:
- Why have you started working with Edword?
- Réka:
- A Refresh Teaching event in September 2019 showed us a couple of educational technology startups, one of which was Easy Correct, the company behind Edword. Their platform offered a way to streamline and optimize the whole feedback flow. Edword was developed jointly by educators and tech personnel, so their insights seemed likely to help teachers improve their feedback’s effectiveness. At first, it was our curiosity that drove us to Edword, but we were quickly convinced to stay with the platform.
- Questioner:
- How did you give feedback before you began using Edword?
- Réka:
- I typically provided feedback electronically in a Word or PDF file (don’t get me started on the inconvenience of that!) or, occasionally, as hard copy. I used a shorthand notation for typical mistakes, as did my colleagues, but it was really time-consuming to explain the details every time to all the students. When I marked up a text, I emailed it to the writer with a few words of overview. All of this took precious time; now I just release the feedback with the click of a button.
- Questioner:
- What difficulties did you encounter in that process?
- Réka:
- Problems ranged from the trivial to the profound. For example, students occasionally didn’t know how to use the “Track changes” function in Word (despite having clear instructions in the handout) and didn’t see any of my comments and changes in the text. This meant they wasted an opportunity to learn – and my time too. In the end, though, the biggest problem was that I didn’t know whether they could make head or tail of my comments. This is where Edword came in.
- Questioner:
- Okay, so now that you’ve been using Edword for a while, what would you say is its biggest advantage for you?
- Réka:
- The fact that we have a much clearer idea how helpful students find our feedback is definitely a game changer. The built-in analytics creates massive opportunities for us to revise our teaching practices on the basis of sound evidence. It’s also a lot easier to include different media in the feedback, like voice recordings, interactive exercises, videos, links, and so on. And let me add quickly that collaborating with my colleagues on comment sets showed me much more about their way of providing feedback than ever before. Besides the efficiency, I think I appreciated this insight into their teaching practice and the increased sense of collegiality the most.
- Questioner:
- I’m sure there were a few limitations, too, weren’t there?
- Réka:
- That’s unavoidable, of course; this is a startup, after all. There were a few bugs and slightly awkward solutions (like having to use Chrome for optimal performance), but once we got the hang of the software, it was pretty intuitive. Whenever we hit a problem, the developers were keen to sort it out; for example, they simplified the downloading of the feedback in PDFs, and when we asked for more detailed rubrics for grading, they quickly made these available.
- Questioner:
- Let me backtrack for a moment: you mentioned “getting the hang of it”. How easy was it to get started?
- Réka:
- To be frank, it does need a bit of time at the start to produce a core comment set. I created my first comment set with about 20 or 30 comments before the start of the semester. With that in my pocket, I could start work, and I kept on adding comments to this set throughout the semester. Now, I have 79 comments in my main comment set, but others work with more or less. It really depends on your own teaching style and priorities.
- Questioner:
- Have you helped others with the onboarding?
- Réka:
- It was mostly done by Edword folks, but I also helped smooth out a few wrinkles. What proved most effective was our shared comment set: with a few examples to go by, most colleagues could pick up the thread rather quickly.
- Questioner:
- How have students responded to Edword?
- Réka:
- Their reaction has been overwhelmingly positive. When asked if they prefer feedback in Edword or by any other means, 87% opted for Edword. I think that says it all.
- Questioner:
- Have you considered who at ETH could benefit from using Edword?
- Réka:
- One course participant answered that question like this: “any kind of course where a written assignment has to be handed in (so I guess a lot ;).” I agree: this tool looks likely to increase the effectiveness of feedback whatever the discipline or subject of instruction.
Learning more about Edwords
Interested in learning more about Edword?
Visit our Moodle page and ask your questions on the forum.
Interested in Testing Edword?
LET is currently planning a second pilot of Edword to trial its usability across the disciplines and to test its Moodle integration. If you are interested in participating, please contact Ms. Melanie Walter at melanie.walter@let.ethz.ch.
