Open Assessment Bank: FullMarks well underway

I’ve mentioned wanting to develop an open assessment bank to complement my other projects and that it was recently approved. I’m happy to report that development is well underway. We are calling this project FullMarks.

The logo and website are still pending and, of course, we’re crowdsourcing their design. Feel free to take a look at 99Designs where we’ve just opened the competition.

We are extending the work done by the Human Sciences Research Council (HSRC) on a closed bank (but open source software). We have the blessing of the HSRC to proceed but, in the interests of full disclosure, with a strong word of caution that an assessment bank can be a blunt intstrument. A bank will save teachers time and increase the average quality of items but their research has shown that, unless teachers act on the identification of learners’ strengths and weaknesses, little real impact is felt when considering learners’ understanding.

I’m happy to charge on ahead with development of this particular blunt instrument, not because I’m irresponsible (trust me, if I wanted to be irresponsible I wouldn’t give myself another project to worry about) but because I feel that their concerns provide the very justification for the open bank. I’ll have a crack at a brief explanation of why I think this is so.

Saving Teachers’ Time

Have you spoken to a teacher in South Africa lately? Time is their biggest concern. Now I expect to have some people disagree with that, they’ll cite resources, lack of support etc. but the very first thing teachers tell me is that they don’t have enough time. Sure, I hear about resources, poor support, curriculum obfuscation, and all sorts of other things, but time is always first.

This isn’t a trick to get teachers’ attention, there is no question that time is an issue so we should work to save them time if we can. The time saving decreases stress and leads to focus that can be deployed elsewhere.

Improved Quality of Assessment Items

We know that the sharing of assessment items can lead to an increase in the average quality of items used in the classroom. Again, there is no question that this is a good thing. As the number quality items grows, teachers will need to develop fewer questions, allowing them to spend more time being creative and focusing on quality rather than quantity.

Open Assessment Bank

Just saving time and improving quality can still lead to little impact on the learner as the HSRC point out. In their analysis, however, they’ve been looking at a closed bank where teachers are passive consumers. My bank will be an open bank allowing questions to be viewed, added, duplicated, and adapted. We will also allow for feedback in a few ways. Users will be able to vote as to whether or not a question’s solution is correct, they will be able to rate questions and they will be able to comment on questions and answers.

Feedback is the key to making sure that the assessment bank is an effective tool (as opposed to a blunt instrument). A lot can be learnt from receiving feedback from your peers as to whether or not the solutions to your questions are correct. Even more can be learnt from a detailed discussion around whether or not a question tests the assessment standards that it is tagged with and how to respond if learners struggle with the question. This will also help teachers further their understanding of the outcomes and appropriate assessment of them. Teachers just need to participate in the community, sharing items, ideas and feedback.

The final possibility of improved understanding of the outcomes and how to respond to learners’ responses, correct and incorrect, is what gives FullMarks so much more potential impact. It is the open aspect that is crucial. My job is to create an environment where teachers feel there is real benefit to contributing. At first this will be to save time, but over time it will evolve to active participation in discussions around how to assess according to the standards and respond when learners struggle.