I’ve just completed an updated, and hopefully, improved presentation on the functionality of the FullMarks website. It takes the form of a walk-through tutorial demonstrating how simple the site is to use. I don’t think any further explanation is required, please take a look and if you like it spread the word about the existence of FullMarks:

On the 24th of March we held the very first FullMarks event, an uploading sprint. The site was, and still is, in beta and the event was the first opportunity to let a random selection of participants have unrestricted access. All previous site demonstrations had been under quite controlled conditions and much smaller.

The event was held at the University of Cape Town‘s Chemical Engineering Department and ran for the whole day. You can see the invitation and agenda here. We had 30 participants supported by the actual software development team from Upfront Systems. The objective was to start to enter content into the assessment bank in preparation for an official launch later this year. We were able to use one of the computer laboratories in the department and were to use the university’s internet connection. The event was held during the university holiday which simplified logistics significantly.

All of the participants brought content with them either on laptop, CD or memory stick. We began the day with an introduction to Open Educational Resources and their benefits which was well received.

Then we began with the work. We allowed the participants to begin by creating questions directly on the site using the creation form so that they would understand the structure of what was being stored before moving on to uploading using the OpenOffice importer functionality. The bulk of the day was spent on uploading.

Being the first users they couldn’t yet take advantage of the test creation functionality as the bank was empty. The last session of the day was used to then demonstrate the test creation and analysis functionality.

The spectrum of teachers ensured that we had a variety of grades covered, many different subjects/learning areas and that we had a content in many different formats, digital and layout, that people wanted to import which gave us a great opportunity to stress test the question creation and importing functionality as well as test creation.

Upfront Systems were on hand to resolve a number of small issues identified as well as discuss educators’ needs with them directly. The long-term success of FullMarks will depend on ensuring the link between the development team and educators remains as direct, clear and effective as possible. Some more serious issues were also discovered which is great for developing a robust solution and having the users and developers in the room meant that the issues were quickly pinned down, without the long discussions around reproducibility that sometimes happen online.

The most exciting part of the day was the willingness of the teachers to share material and the opportunity to connect them with the Upfront team directly. It is important to get questions into the bank but strengthening the community around the project will really have long-term benefits for the project.

We had intended to run 6 workshops at the AMESA Congress the starting the Sunday after this sprint but one of the issues identified, involving non-standard styling comments from Word, was able to cripple the site by throwing the PDF generator into an infinite loop, and any user could do it set it off. This was discovered just before the workshop when someone did it and so the FullMarks workshops at the AMESA Congress became Siyavula workshops instead.

We’re working on an alternative plan to ramp up our content and user-base before the official launch of FullMarks, watch this space for more details.

The photographs are courtesy of René Toerien and can be found here.

Right click on image and select “view image” to see larger versions.

I have made a quick slideshow tour of some of the functionality of the alpha version of the FullMarks site. The site is extremely simple yet offers powerful reports to help support learners effectively, especially in the current environment in South Africa where classes are large and teachers don’t have enough time to consider individual needs.

I’ve been doing quite a lot of work on the FullMarks front, much more than has been reported here. An alpha version of the site has been available for a while now and I’ve been using it to gather support for the FullMarks project launch. The official launch will take place after an uploading sprint.

The purpose of the sprint is to ensure that at the site isn’t empty at the launch event which would disappoint everyone and knock the project’s momentum. Participants in the sprint aren’t expecting a website full of content so there will be no disappointment there. They will know that they are leading the way in setting up a site that captures the attention of many other teachers, something they will benefit from as users of FullMarks, when it is launched.

Having an upload sprint means that the relatively mundane task of uploading thousands of questions into FullMarks becomes a shared burden in a very tangible way, it focuses the effort into a single day and makes the process a much more sociable exercise. In this way, not only are teachers sharing with each other in a structured, categorised and efficient way, and laying a platform to help many isolated and under-resourced teachers, they are also strengthening the teaching community.

I’ve approached a number of schools about an opportunity to present the project to a representative of the school and have so far visited (in order of first visit):

• Westerford High School
• Pinelands North Primary School
• Wynberg Boys’ High School
• Wynberg Girls’ High School
• Pinelands High School

The reason I am approaching schools first rather than subject focus groups is that I want the maximum breadth of questions entered into FullMarks so that it is of interest to the widest possible spectrum of teachers. Sufficient schools participating will give the bank the depth it needs. I will definitely still be approaching the subject focus groups that I know of once I have confirmed the participation of a number of schools.

I have had the opportunity to return to Wynberg Girls’ High School and Pinelands North Primary School to demonstrate the scope of functionality of the alpha version of the site to interested members of staff. At both schools the principals have sanctioned the participation of their teachers in the sprint. However, I still have to convince teachers to participate. Pinelands North Primary is the first school to go the next step and to confirm that they have a set of teachers who will participate which I am very excited about.

I will be approaching more schools in the coming weeks and contacting the mathematics and physical science swap and share groups that I became aware of through the Siyavula project. My ambitious goal is 150 participants in the upload sprint and hopefully 10 000 questions. That would get the project off to a flying start!

Upfront Systems, the team that is developing FullMarks, also confirmed today that they will provide the technical support at the sprint event which is great because they know the site better than anyone. It will also help future development of the site because it will give teachers the opportunity to talk directly to the developers making sure everyone is on the same page. It is crucial that FullMarks be aligned with teachers needs as well as being as easy to use as possible so that it gains real traction and has a positive impact in South Africa.

Things are really starting to fall into place for FullMarks and the next 6 weeks will be very exciting for this project.

If you are reading this and you or your school are interested in a demo just leave a comment or send me an email and I’d be more than happy to show you the site and give you more details about the project.

The development of the FullMarks open assessment bank is well underway. We’re on track for a launch early next year. Without a beta version of the site up it is hard to show you the progress at this time but we’re about 20 man-days away from the website being ready to show you.

We have got the look of the front-page tied down though and here is a screenshot of what www.fullmarks.org.za will look like early in 2010:

Once the site is up we’ll be organising some content uploading sprints to make sure we get the breadth of subjects covered as quickly as possible.

I’ve mentioned wanting to develop an open assessment bank to complement my other projects and that it was recently approved. I’m happy to report that development is well underway. We are calling this project FullMarks.

The logo and website are still pending and, of course, we’re crowdsourcing their design. Feel free to take a look at 99Designs where we’ve just opened the competition.

We are extending the work done by the Human Sciences Research Council (HSRC) on a closed bank (but open source software). We have the blessing of the HSRC to proceed but, in the interests of full disclosure, with a strong word of caution that an assessment bank can be a blunt intstrument. A bank will save teachers time and increase the average quality of items but their research has shown that, unless teachers act on the identification of learners’ strengths and weaknesses, little real impact is felt when considering learners’ understanding.

I’m happy to charge on ahead with development of this particular blunt instrument, not because I’m irresponsible (trust me, if I wanted to be irresponsible I wouldn’t give myself another project to worry about) but because I feel that their concerns provide the very justification for the open bank. I’ll have a crack at a brief explanation of why I think this is so.

Saving Teachers’ Time

Have you spoken to a teacher in South Africa lately? Time is their biggest concern. Now I expect to have some people disagree with that, they’ll cite resources, lack of support etc. but the very first thing teachers tell me is that they don’t have enough time. Sure, I hear about resources, poor support, curriculum obfuscation, and all sorts of other things, but time is always first.

This isn’t a trick to get teachers’ attention, there is no question that time is an issue so we should work to save them time if we can. The time saving decreases stress and leads to focus that can be deployed elsewhere.

Improved Quality of Assessment Items

We know that the sharing of assessment items can lead to an increase in the average quality of items used in the classroom. Again, there is no question that this is a good thing. As the number quality items grows, teachers will need to develop fewer questions, allowing them to spend more time being creative and focusing on quality rather than quantity.

Open Assessment Bank

Just saving time and improving quality can still lead to little impact on the learner as the HSRC point out. In their analysis, however, they’ve been looking at a closed bank where teachers are passive consumers. My bank will be an open bank allowing questions to be viewed, added, duplicated, and adapted. We will also allow for feedback in a few ways. Users will be able to vote as to whether or not a question’s solution is correct, they will be able to rate questions and they will be able to comment on questions and answers.

Feedback is the key to making sure that the assessment bank is an effective tool (as opposed to a blunt instrument). A lot can be learnt from receiving feedback from your peers as to whether or not the solutions to your questions are correct. Even more can be learnt from a detailed discussion around whether or not a question tests the assessment standards that it is tagged with and how to respond if learners struggle with the question. This will also help teachers further their understanding of the outcomes and appropriate assessment of them. Teachers just need to participate in the community, sharing items, ideas and feedback.

The final possibility of improved understanding of the outcomes and how to respond to learners’ responses, correct and incorrect, is what gives FullMarks so much more potential impact. It is the open aspect that is crucial. My job is to create an environment where teachers feel there is real benefit to contributing. At first this will be to save time, but over time it will evolve to active participation in discussions around how to assess according to the standards and respond when learners struggle.