Today, I’m excited to say, we had our 1000th user sign up for FullMarks. There are still about 340 000 to go in South Africa but I’m still very excited about this milestone. We’re edging slowly towards that critical mass of users that start to interact with each other. I can’t say I know what the magical number will be but 1000 is closer to it than 500 :)

We also want to get our users much more involved in the project as a whole. It was built for them, not for us, so we need to know what they love, like, hate and want so that we can ensure the project evolves to meet their needs as efficiently and effectively as possible.

To make them, and everyone for that matter, more aware of what is happening with FullMarks we’ve launched a blog specifically for the project – please take a look at http://fullmarks.siyavula.org.za. We’re specifically looking for input on our roadmap for the next 6 months.

I’ll still post personal thoughts about the project here but news, events and roadmaps will be posted on the project blog.

Today the FullMarks website achieved a significant milestone. The numbers may appear modest in the context of huge sites like Facebook and Flickr but in the South African context I think they are significant. Within a 12 hour period we passed both 500 users and 5000 items on FullMarks and I’m excluding demo users and content in that count so it represents real usage.

In South Africa approximately 12% of people have internet access and we are targeting educators (a pool of about 400 000 people in total). The total pool means that there is massive potential for FullMarks if we can tap into a the 40 000 that currently have internet access. There are many initiatives to give more educators access so this number should grow relatively rapidly, especially with the costs of bandwidth coming down and the low price of entry-level laptops and netbooks. Content consumption isn’t really the interesting part and I think that many will look and use but even if only 1% can be convinced to contribute back the site will do well. 400 active contributors would be significant.

We don’t know at what point the community will really take off and content production will not require any further work on our part. We are still running workshops and training new people on how the site works. However, we do not run workshops every day and we do get new users every day. We are still supporting some content uploading so content is not entirely from the community yet.

Here is an animation made from our Google Analytics for the site that shows where in South Africa traffic is coming from:

I think that this is the first indication of some viral spread. The key for us now is to create an environment where this distributed group feel that participation is worthwhile and that they spread the word.

The Cape Town Siyavula team was met with wintry weather in Durban as we touched down early on Friday morning. This did not dampen our spirits as we made our way to the Southern Sun North Beach Hotel to set up for the workshop. With a turnout of 21 AMESA teachers, we were A-for away and the workshop began.

As often happens when one has a set agenda, we veered off course straight away and got stuck into building topic trees, for assigned school Grades. Amidst much discussion the topic trees emerged, and it was then felt that a break for tea was well deserved. This was followed by an interesting talk by Prof Michael de Villiers of UKZN, who spoke about maths and symmetry in everyday life. As early evening approached, Mark initiated the introduction to uploading questions to the FullMarks website (agenda item 1!). He also demonstrated how to create equations using MathML, and how to take screenshots to upload diagrams.

This brought us to 19h10 which meant we had to hot foot it to uShaka for our dinner reservation at the Cargo Hold. There we spent an enjoyable evening getting to know one another and watching the fish and sharks swimming peacefully past us in the aquarium, as our 3 course meal was served. Despite our previous threats that the slow uploaders or those who misbehaved may end up being shark food, I am pleased to announce that everyone behaved themselves and so were spared this terrifying fate.

We were greeted by a beautiful summer’s day on Day Two, which began with a lovely breakfast buffet in the dining room of the hotel. The workshop then kicked off just after 09h00, with the goal for the day being to upload as many questions as possible.This had also been the goal from Day One which unfortunately was not realised, but we were determined to see success by the end of the afternoon. With a quick revision to start with, and internet voucher numbers entered, the teachers were ready to go.

The Siyavula team were on hand to provide tech support to the uploaders, especially as they attempted to use the maths editor on the FullMarks website. The maths editor is not particularly difficult to work with, it just takes a bit of practice to understand how to enter the data. It didn’t take long before everyone was getting the hang of things, and so the team agreed it was time to throw a spanner in the works and show the AMESA teachers a new system for uploading questions and answers!This was presented to them in the form of an OpenOffice template, which Mark promised was a very simple and far more efficient system to use. A fairly quick demonstration was given, which was greeted with exclamations of “why didn’t you show us this from the start?” and “this is so much easier than the other way!”. So much for being worried about confusing everyone further!

We had a very productive afternoon as the teachers’ confidence increased as they practiced using the OpenOffice template, and uploaded multiple questions and answers to the FullMarks website. As the sounds of success echoed around the room, another tea break was due and the chance to discuss our learnings amongst ourselves. It’s amazing what milktart and tea can do for conversation!

That was to be the final feeding and watering of the troops, and so once that was over Mark began the wrap up of the workshop weekend. Tasks were assigned to different teachers, and a discussion took place over what the next step would be. With tentative plans in place for the next workshop, everyone went their separate ways amidst smiles and hugs, and the Siyavula team was left feeling satisfied at the knowledge that was imparted and the progress that was made.

The Southern Sun North Beach Hotel provided a great venue for the workshop, despite the issues with holding an online workshop using internet vouchers. The vouchers had to be purchased in denominations of minutes to spend online, as opposed to MB or GB usage. It didn’t take long before vouchers were expiring and thus logging the users out of the FullMarks page, usually mid-upload; or if the page was left idle for too long, the same thing would happen. This hindered progress somewhat and caused frustration, especially as the uploading process is relatively painless and straightforward. However, the teachers were very patient, and understood that when they upload from a stable line, they will not have this problem.

Overall the Siyavula team had a great weekend of meeting new people, imparting knowledge, eating far too much delicious food, and most likely not getting enough sleep! Well done to everyone that attended and to the Siyavula team for organising this great event.

I’ve just completed an updated, and hopefully, improved presentation on the functionality of the FullMarks website. It takes the form of a walk-through tutorial demonstrating how simple the site is to use. I don’t think any further explanation is required, please take a look and if you like it spread the word about the existence of FullMarks:

On the 24th of March we held the very first FullMarks event, an uploading sprint. The site was, and still is, in beta and the event was the first opportunity to let a random selection of participants have unrestricted access. All previous site demonstrations had been under quite controlled conditions and much smaller.

The event was held at the University of Cape Town‘s Chemical Engineering Department and ran for the whole day. You can see the invitation and agenda here. We had 30 participants supported by the actual software development team from Upfront Systems. The objective was to start to enter content into the assessment bank in preparation for an official launch later this year. We were able to use one of the computer laboratories in the department and were to use the university’s internet connection. The event was held during the university holiday which simplified logistics significantly.

All of the participants brought content with them either on laptop, CD or memory stick. We began the day with an introduction to Open Educational Resources and their benefits which was well received.

Then we began with the work. We allowed the participants to begin by creating questions directly on the site using the creation form so that they would understand the structure of what was being stored before moving on to uploading using the OpenOffice importer functionality. The bulk of the day was spent on uploading.

Being the first users they couldn’t yet take advantage of the test creation functionality as the bank was empty. The last session of the day was used to then demonstrate the test creation and analysis functionality.

The spectrum of teachers ensured that we had a variety of grades covered, many different subjects/learning areas and that we had a content in many different formats, digital and layout, that people wanted to import which gave us a great opportunity to stress test the question creation and importing functionality as well as test creation.

Upfront Systems were on hand to resolve a number of small issues identified as well as discuss educators’ needs with them directly. The long-term success of FullMarks will depend on ensuring the link between the development team and educators remains as direct, clear and effective as possible. Some more serious issues were also discovered which is great for developing a robust solution and having the users and developers in the room meant that the issues were quickly pinned down, without the long discussions around reproducibility that sometimes happen online.

The most exciting part of the day was the willingness of the teachers to share material and the opportunity to connect them with the Upfront team directly. It is important to get questions into the bank but strengthening the community around the project will really have long-term benefits for the project.

We had intended to run 6 workshops at the AMESA Congress the starting the Sunday after this sprint but one of the issues identified, involving non-standard styling comments from Word, was able to cripple the site by throwing the PDF generator into an infinite loop, and any user could do it set it off. This was discovered just before the workshop when someone did it and so the FullMarks workshops at the AMESA Congress became Siyavula workshops instead.

We’re working on an alternative plan to ramp up our content and user-base before the official launch of FullMarks, watch this space for more details.

The photographs are courtesy of René Toerien and can be found here.

Right click on image and select “view image” to see larger versions.

I have made a quick slideshow tour of some of the functionality of the alpha version of the FullMarks site. The site is extremely simple yet offers powerful reports to help support learners effectively, especially in the current environment in South Africa where classes are large and teachers don’t have enough time to consider individual needs.

I’ve been doing quite a lot of work on the FullMarks front, much more than has been reported here. An alpha version of the site has been available for a while now and I’ve been using it to gather support for the FullMarks project launch. The official launch will take place after an uploading sprint.

The purpose of the sprint is to ensure that at the site isn’t empty at the launch event which would disappoint everyone and knock the project’s momentum. Participants in the sprint aren’t expecting a website full of content so there will be no disappointment there. They will know that they are leading the way in setting up a site that captures the attention of many other teachers, something they will benefit from as users of FullMarks, when it is launched.

Having an upload sprint means that the relatively mundane task of uploading thousands of questions into FullMarks becomes a shared burden in a very tangible way, it focuses the effort into a single day and makes the process a much more sociable exercise. In this way, not only are teachers sharing with each other in a structured, categorised and efficient way, and laying a platform to help many isolated and under-resourced teachers, they are also strengthening the teaching community.

I’ve approached a number of schools about an opportunity to present the project to a representative of the school and have so far visited (in order of first visit):

• Westerford High School
• Pinelands North Primary School
• Wynberg Boys’ High School
• Wynberg Girls’ High School
• Pinelands High School

The reason I am approaching schools first rather than subject focus groups is that I want the maximum breadth of questions entered into FullMarks so that it is of interest to the widest possible spectrum of teachers. Sufficient schools participating will give the bank the depth it needs. I will definitely still be approaching the subject focus groups that I know of once I have confirmed the participation of a number of schools.

I have had the opportunity to return to Wynberg Girls’ High School and Pinelands North Primary School to demonstrate the scope of functionality of the alpha version of the site to interested members of staff. At both schools the principals have sanctioned the participation of their teachers in the sprint. However, I still have to convince teachers to participate. Pinelands North Primary is the first school to go the next step and to confirm that they have a set of teachers who will participate which I am very excited about.

I will be approaching more schools in the coming weeks and contacting the mathematics and physical science swap and share groups that I became aware of through the Siyavula project. My ambitious goal is 150 participants in the upload sprint and hopefully 10 000 questions. That would get the project off to a flying start!

Upfront Systems, the team that is developing FullMarks, also confirmed today that they will provide the technical support at the sprint event which is great because they know the site better than anyone. It will also help future development of the site because it will give teachers the opportunity to talk directly to the developers making sure everyone is on the same page. It is crucial that FullMarks be aligned with teachers needs as well as being as easy to use as possible so that it gains real traction and has a positive impact in South Africa.

Things are really starting to fall into place for FullMarks and the next 6 weeks will be very exciting for this project.

If you are reading this and you or your school are interested in a demo just leave a comment or send me an email and I’d be more than happy to show you the site and give you more details about the project.

The development of the FullMarks open assessment bank is well underway. We’re on track for a launch early next year. Without a beta version of the site up it is hard to show you the progress at this time but we’re about 20 man-days away from the website being ready to show you.

We have got the look of the front-page tied down though and here is a screenshot of what www.fullmarks.org.za will look like early in 2010:

Once the site is up we’ll be organising some content uploading sprints to make sure we get the breadth of subjects covered as quickly as possible.

I’ve mentioned wanting to develop an open assessment bank to complement my other projects and that it was recently approved. I’m happy to report that development is well underway. We are calling this project FullMarks.

The logo and website are still pending and, of course, we’re crowdsourcing their design. Feel free to take a look at 99Designs where we’ve just opened the competition.

We are extending the work done by the Human Sciences Research Council (HSRC) on a closed bank (but open source software). We have the blessing of the HSRC to proceed but, in the interests of full disclosure, with a strong word of caution that an assessment bank can be a blunt intstrument. A bank will save teachers time and increase the average quality of items but their research has shown that, unless teachers act on the identification of learners’ strengths and weaknesses, little real impact is felt when considering learners’ understanding.

I’m happy to charge on ahead with development of this particular blunt instrument, not because I’m irresponsible (trust me, if I wanted to be irresponsible I wouldn’t give myself another project to worry about) but because I feel that their concerns provide the very justification for the open bank. I’ll have a crack at a brief explanation of why I think this is so.

Saving Teachers’ Time

Have you spoken to a teacher in South Africa lately? Time is their biggest concern. Now I expect to have some people disagree with that, they’ll cite resources, lack of support etc. but the very first thing teachers tell me is that they don’t have enough time. Sure, I hear about resources, poor support, curriculum obfuscation, and all sorts of other things, but time is always first.

This isn’t a trick to get teachers’ attention, there is no question that time is an issue so we should work to save them time if we can. The time saving decreases stress and leads to focus that can be deployed elsewhere.

Improved Quality of Assessment Items

We know that the sharing of assessment items can lead to an increase in the average quality of items used in the classroom. Again, there is no question that this is a good thing. As the number quality items grows, teachers will need to develop fewer questions, allowing them to spend more time being creative and focusing on quality rather than quantity.

Open Assessment Bank

Just saving time and improving quality can still lead to little impact on the learner as the HSRC point out. In their analysis, however, they’ve been looking at a closed bank where teachers are passive consumers. My bank will be an open bank allowing questions to be viewed, added, duplicated, and adapted. We will also allow for feedback in a few ways. Users will be able to vote as to whether or not a question’s solution is correct, they will be able to rate questions and they will be able to comment on questions and answers.

Feedback is the key to making sure that the assessment bank is an effective tool (as opposed to a blunt instrument). A lot can be learnt from receiving feedback from your peers as to whether or not the solutions to your questions are correct. Even more can be learnt from a detailed discussion around whether or not a question tests the assessment standards that it is tagged with and how to respond if learners struggle with the question. This will also help teachers further their understanding of the outcomes and appropriate assessment of them. Teachers just need to participate in the community, sharing items, ideas and feedback.

The final possibility of improved understanding of the outcomes and how to respond to learners’ responses, correct and incorrect, is what gives FullMarks so much more potential impact. It is the open aspect that is crucial. My job is to create an environment where teachers feel there is real benefit to contributing. At first this will be to save time, but over time it will evolve to active participation in discussions around how to assess according to the standards and respond when learners struggle.