I just wanted to quickly dump a few thoughts about a big event we’re about to run. I’ve been slipping behind in my blogging and thought I’d inflict a quick core dump on you.

Tomorrow (7^th May 2011) we are holding a translation hackathon at the University of Stellenbosch. The objective is extremely ambitious, translate the Grade 10 Mathematics and Physical Science FHSST texts into Afrikaans in one day. We’ve been discussing this possibility for a long time and we finally thought we’d just have a crack at it.

The pressure had been mounting after we were given some airtime on an Afrikaans radio station, Radio Sonder Grense. We were contacted by a number of people wanting to help translate. We had 3 people start doing this virtually but we didn’t make much progress. Most were unfamiliar with the platform and the books and it wasn’t our focus so they got a little lost and gave up. It allowed to start thinking about the process and begin preparing some documentation and workgroups for translation on Connexions.

Then we finally had someone volunteer who was familiar with markup languages and felt comfortable translating the raw Connexions XML, Carl Scheffler. Carl translated 3 chapters into Afrikaans and gave us the first real estimates for how long a chapter should take someone who is comfortable looking at a markup language and confident to do the translation, our average chapter length should take one man day for a first pass translation.

This was the information we needed to proceed, we contact Thinus Booysen, a programming lecturer at Stellenbosch just to see if he’d be prepared to mention it to his class. Thinus jumped straight in and practically organised the participants, formal approval and a venue. So the die was cast!

Given our experiences with regular hackathons, the failed translation attempts and the fact that the translation hackathon is to be a completely new process we’ve tried to put some things into place to mitigate the risk of failure but also ensure that the event has the most meaning for the participants. It is a once-off event so we must get it right.

Event Structure

We know that we need to process people very effectively when they arrive. I’m sure this is obvious but still a challenge, everyone needs extremely clear, simple instructions; they need to be quickly and simply made aware of all the necessary processes, structures and they need to be made as comfortable as possible.

The layout of the room and other visual cues assist in this. They must be channeled to one person who makes sure they’re made to feel welcome, relevant and able to participate. The team must be either introduced to everyone (possibly impractical) or very easily identifiable.

We’re not going to begin with a presentation but actually create a sprint-like working atmosphere to get the ball rolling. The introductory presentation will actually be delay until 1.5 hours after start time by which time everyone should definitely have started working. It seems that if people have to wait until a critical mass has arrived and then sit through a presentation, the uncertainty about the actual process (regardless of the presentation info) builds up and people become frustrated. So our refined instructions should allow everyone to get started on actual translation work without a presentation or much background information.

Community Aspects

We want people, many of whom have never met, to feel as much part of a team and like they are achieving something. To help people get a sense of the team and its progress we’ve developed a small tool to produce a webpage that iterates through all chapters highlighting with a progress bar the state and showing a photograph and name of the translator.

This will be projected on the screen and is linked to a progress spreadsheet which will be continuously updated. Every time it is changed a small script will update the webpage. Anyone will be able to see at any given time how we are making progress as a team who is working on what chapter. This is useful because many chapters build on concepts etc. from previous chapters so there will need to be interaction between translators to answer questions like which word was used for this, how precisely was this defined etc.

The progress webpage will be live here from 9am (GMT+02:00) tomorrow.

Focus Groups

One thing we know is no matter how well prepared we think we are things can go wrong, so we tested our process with a small focus group. The focus group lasted an afternoon and had people who had never used the site before involved. It gave us a great perspective on how ready we were and how volunteers would behave.

One area we’ve had some problems is the performance of the site, this has led to us prototyping other editors etc. We tried to commission a lightweight editor that would allow translator to only edit the proper text and forbid changes to the markup. Unfortunately this hasn’t been completed in time. This is the one tool that could really streamline translation and we feel that it is worth building even after our event is complete.

Risk

There are two primary risks, both technical. There is a very real possibility we’ll have editing issues on the site (there seems to be a dodgy proxy-server between us and Connexions that regularly caches a corrupt javascript file and it doesn’t respect refresh requests). To mitigate this I will download all the modules from Connexions early tomorrow morning and people can just translate in any text editor.

The other risk has to do with technical aspects related to the language used. To mitigate this we do have a number of Mathematics and Science educators coming who teach in Afrikaans. It will be important to iterate rapidly in the first hour having them give each translator feedback on a section or two to get them into the right groove.

Anticipated Outcomes

In addition to translating a lot of content we hope to create a mini-documentary video that highlights both the process, the successes and the volunteers who made it possible. We’ll be doing a video interview with everyone who is willing and then try to put up a video in the next week.

If all goes well we’ll have two Afrikaans textbooks next week and a video of how it all came about.

On the 24th of March we held the very first FullMarks event, an uploading sprint. The site was, and still is, in beta and the event was the first opportunity to let a random selection of participants have unrestricted access. All previous site demonstrations had been under quite controlled conditions and much smaller.

The event was held at the University of Cape Town‘s Chemical Engineering Department and ran for the whole day. You can see the invitation and agenda here. We had 30 participants supported by the actual software development team from Upfront Systems. The objective was to start to enter content into the assessment bank in preparation for an official launch later this year. We were able to use one of the computer laboratories in the department and were to use the university’s internet connection. The event was held during the university holiday which simplified logistics significantly.

All of the participants brought content with them either on laptop, CD or memory stick. We began the day with an introduction to Open Educational Resources and their benefits which was well received.

Then we began with the work. We allowed the participants to begin by creating questions directly on the site using the creation form so that they would understand the structure of what was being stored before moving on to uploading using the OpenOffice importer functionality. The bulk of the day was spent on uploading.

Being the first users they couldn’t yet take advantage of the test creation functionality as the bank was empty. The last session of the day was used to then demonstrate the test creation and analysis functionality.

The spectrum of teachers ensured that we had a variety of grades covered, many different subjects/learning areas and that we had a content in many different formats, digital and layout, that people wanted to import which gave us a great opportunity to stress test the question creation and importing functionality as well as test creation.

Upfront Systems were on hand to resolve a number of small issues identified as well as discuss educators’ needs with them directly. The long-term success of FullMarks will depend on ensuring the link between the development team and educators remains as direct, clear and effective as possible. Some more serious issues were also discovered which is great for developing a robust solution and having the users and developers in the room meant that the issues were quickly pinned down, without the long discussions around reproducibility that sometimes happen online.

The most exciting part of the day was the willingness of the teachers to share material and the opportunity to connect them with the Upfront team directly. It is important to get questions into the bank but strengthening the community around the project will really have long-term benefits for the project.

We had intended to run 6 workshops at the AMESA Congress the starting the Sunday after this sprint but one of the issues identified, involving non-standard styling comments from Word, was able to cripple the site by throwing the PDF generator into an infinite loop, and any user could do it set it off. This was discovered just before the workshop when someone did it and so the FullMarks workshops at the AMESA Congress became Siyavula workshops instead.

We’re working on an alternative plan to ramp up our content and user-base before the official launch of FullMarks, watch this space for more details.

The photographs are courtesy of René Toerien and can be found here.

I’ve been doing quite a lot of work on the FullMarks front, much more than has been reported here. An alpha version of the site has been available for a while now and I’ve been using it to gather support for the FullMarks project launch. The official launch will take place after an uploading sprint.

The purpose of the sprint is to ensure that at the site isn’t empty at the launch event which would disappoint everyone and knock the project’s momentum. Participants in the sprint aren’t expecting a website full of content so there will be no disappointment there. They will know that they are leading the way in setting up a site that captures the attention of many other teachers, something they will benefit from as users of FullMarks, when it is launched.

Having an upload sprint means that the relatively mundane task of uploading thousands of questions into FullMarks becomes a shared burden in a very tangible way, it focuses the effort into a single day and makes the process a much more sociable exercise. In this way, not only are teachers sharing with each other in a structured, categorised and efficient way, and laying a platform to help many isolated and under-resourced teachers, they are also strengthening the teaching community.

I’ve approached a number of schools about an opportunity to present the project to a representative of the school and have so far visited (in order of first visit):

• Westerford High School
• Pinelands North Primary School
• Wynberg Boys’ High School
• Wynberg Girls’ High School
• Pinelands High School

The reason I am approaching schools first rather than subject focus groups is that I want the maximum breadth of questions entered into FullMarks so that it is of interest to the widest possible spectrum of teachers. Sufficient schools participating will give the bank the depth it needs. I will definitely still be approaching the subject focus groups that I know of once I have confirmed the participation of a number of schools.

I have had the opportunity to return to Wynberg Girls’ High School and Pinelands North Primary School to demonstrate the scope of functionality of the alpha version of the site to interested members of staff. At both schools the principals have sanctioned the participation of their teachers in the sprint. However, I still have to convince teachers to participate. Pinelands North Primary is the first school to go the next step and to confirm that they have a set of teachers who will participate which I am very excited about.

I will be approaching more schools in the coming weeks and contacting the mathematics and physical science swap and share groups that I became aware of through the Siyavula project. My ambitious goal is 150 participants in the upload sprint and hopefully 10 000 questions. That would get the project off to a flying start!

Upfront Systems, the team that is developing FullMarks, also confirmed today that they will provide the technical support at the sprint event which is great because they know the site better than anyone. It will also help future development of the site because it will give teachers the opportunity to talk directly to the developers making sure everyone is on the same page. It is crucial that FullMarks be aligned with teachers needs as well as being as easy to use as possible so that it gains real traction and has a positive impact in South Africa.

Things are really starting to fall into place for FullMarks and the next 6 weeks will be very exciting for this project.

If you are reading this and you or your school are interested in a demo just leave a comment or send me an email and I’d be more than happy to show you the site and give you more details about the project.