The idea of testing crowdsourcing against a more traditional design process came up last year (you can read about it here) and we decided to do an experiment as we needed new covers for the two sets of Free High School Science Texts (FHSST) textbooks.

The crowdsourced covers from 99Designs came in first (featured in this blog post):

Here are the full sets of covers for you to compare (you can leave your opinions about which win in the comments if you like):

Grade 10

Grade 11

Grade 12

Results

To be honest, I prefer the crowdsourced Physical Science covers but, in the interests of full disclosure, I managed that process. I am quite happy with the Mathematics covers. If we had had different designers as part of either process the results could have been different so this is certainly not a definite test with a control group but a little subjectivity makes for better coffee-time debate.

Obviously, the key thing to an end product you can live with is how you manage the process no matter which route you go. I have discovered I am very bad at working with an individual designer on a single project. I find it difficult to articulate what I want and what it is that I like and don’t like. These things make me a difficult client, although I feel this is mitigated slightly by the fact that I’m aware of it.

99Designs competitions are more tolerant of my choices and I feel that some designers seem to be able to figure wear I’m going based on what I rate designs and what designs I eliminate. I’ve started to standardise my own process for running competitions on 99Designs.

Competition

• The competition benefits from designers competing and feeding off each other – if you run a blind contest (where designers cannot see each others entries) you’ll lose out on the complex feedback between different design ideas and will basically be running about 30 individual design processes, requiring a lot more feedback and time. If you’re a bad client (like myself) that struggles to give good feedback this will really be difficult.
• For the first 3 days don’t give a rating higher than 3 stars – no matter how much you like the design. Immediately after launching the competition make a comment to this effect otherwise designers will get unhappy. They’ll grumble but live with it if they know it is your policy. If you don’t comment and do it they’ll just think you’re being difficult. You may well find something you really think you like in the first 48 hours but those strong, early contenders have always faded out as the feedback starts to take place.
• Provide feedback often – everyday – the competition only lasts 7 days but will take a couple of hours a day. If you don’t put effort into getting a design you like why should the designers – they don’t have to live with it.
• Prepay the contest – then 99Designs holds the money – you can still get it back but many designers won’t even consider participating if the competition isn’t prepaid. If you are serious about getting a design and have a budget then prove it.
• Once there are a couple of days left AND some designs you could live with – guarantee the contest – this way you can’t get the money back and must choose a winner – but its an added incentive to the designers to get involved. Many of the best seem to wait until the last 48 hours before entering.
• If designs conflict with the brief – eliminate them – and comment to that effect. Don’t let them hang around. If you hate it – eliminate! The competition feeds on what doesn’t get eliminated and on what gets rated highly.
• Give designers you rate individual feedback on their design – not in the general comments section. Don’t pick an individual designers idea and ask everyone to work on something like that.
• Put general feed back or changes to brief in general comments section.
• Don’t be put out if they moan a bit about your poor design choices, it is your design content after all. If they mail you privately just say that you are following the rules and they should go through the competition page – do NOT argue with them about design, especially not privately. Keep it on the competition page.

Brief (this is for a logo design competition)

Give yourself reasons to eliminate the rubbish easily by including things like:

• no drop shadows
• no gradients
• no 3D effects
• no stock images
• no proprietary fonts (otherwise you have to purchase)
• at most 2 colours (maybe 3
• logo must scale down
• logo must work in B&W
• should be balanced
• colour logo must be presented on white background (if they start
  using textured backgrounds to make the logo look good it is hard to
  compare etc.)

Thats all before you even say anything about what you are looking for. You can easily find much more comprehensive lists for rules for logo design competitions with a quick search online.

Then tell them what you want and include:

• project elevator pitch and link to site or descriptive material if
  you have.

• icon and/or word together
• ideas of colours if you have any
• whether icon should stand on its own
• ideas of fonts

Make sure you ask for:

• Colour version
• B&W version
• Vector files (I’d ask for .ai and .svg files)

I think that crowdsourcing is also a lot more fun than dealing with a single designer and you really do see a lot of different concepts in a very short space of time. I’m sure there is an argument to be made that a longer process might lead a more mature result but I’m sure a counter-argument exists built on intensity and focus for the person running the competition. Ultimately, as with everything in life, you have to make your own decisions and live with them. Luckily taking a chance on a logo for $500 isn’t very risky and worth a shot. The more you pay the more attention you get but for $500 for a logo design I am barely able to keep up with my day job and the competition.

As part of the launch of OpenPress, the FHSST Grade 10 Mathematics book is getting printed. We felt that we needed a fresh set of covers for the books. They will be printed with a colour cover and we felt that the all the Maths covers should have a theme; similarly for the Physical Science books.

The covers will need to be available in the next 10 days or so and we were discussing the best way to get them done. The obvious way to go is to commission the work from a designer and go with that. We also considered the option of crowdsourcing the covers and have had a lot of debate about the merits of crowdsourcing.

To resolve the issue, once and for all (maybe), we’ve decided to run our own internal competition. The prize will be bragging rights as the optimal process for future covers for OpenPress / FHSST texts. We are going to allocate the same budget to the Maths and Physical Science covers, but we are going to have the Maths set commissioned by a regular design firm and we’ll get the Physical Science ones created on 99Designs.

Although it isn’t really a competition it will be fun to see the results. The only way to keep the results comparable is to give both processes the same:

• budget,
• brief,
• quantity and quality of feedback, and
• the same timeline.

I’ll post the brief a little later and the results as well so you can decide which you think is the best set and whether we’ve chosen the best process based on this little experiment. If nothing else this should be fun.

The Sunday Times newspaper commissioned a study of South African schools which they released this last weekend. There is a lot of information in the report and one could spend a lot of time unpacking it. The Sunday Times chose to highlight some things that would definitely lead to a fair amount of debate, some of the classic ones being:

• same-sex versus co-ed schools, and
• girls being smarter than boys.

I’m not really very interested in venturing down either of those paths at this time, or at any time for that matter.

The report is worth a couple of blog posts but I’d like to start by just focusing on one school that was mentioned. Mbilwi Secondary School was ranked in the survey as the top science school in the country. It is a school that I have never heard and, I am quite sure, many other people had never heard of. I haven’t done any additional research on Mbilwi … yet, but there are somethings that jump out at me from the report:

• Mbilwi has large classes, and
• students have to share study guides.

Why do those fact jump out? Large classes and a lack of resources are often cited reasons for poor preformance at schools. Reading a little further they point out just how large the classes are (pupil:teacher ratio of 56:1) and that the school attributes their success to:

• identifying weak learners at the beginning of matric and providing extra support 4 days a week right up until the final exams,
• extra tuition on Saturdays, and
• focusing on more than Grade 12 but working to help learners from Grade 8.

These are the reported statistics that allowed them to achieve the top spot (Matric 2008 numbers):

• 201 Matrics cadidates
• Pass rate 100%
• Univeristy entrance 93%
• 309 A symbols
• 80% passed Maths with more than 50%
• 75% passed Science with more 50%

My take-away from this is that a lot can be done with the right attitude. If the learners didn’t want to learn the extra tuition wouldn’t mean anything, if the teachers didn’t care but were forced to provide the extra tuition they’d not do a very good job. The fact that they do it, do it well and that it is well received tells me that everyone involved with Mbilwi has reaslised that the problem is not insurmountable and that hard work can pay off, if everyone does their best working together.

Another point I can’t help but mention,  EVEN THE NUMBER 1 science school in South Africa will benefit from having access to FHSST, Siyavula and OpenPress to solve their resources problems.

To help teachers get the most out of the the full library of Siyavula, Connexions, Free High School Science Texts (FHSST) and many other open textbooks, we want to see these resources printed as cost effectively as possible while still ensuring high print quality. We believe that by aggregating print orders we can do just that. We will facilitate this through an online print aggregation service that we are calling OpenPress.

Our vision for OpenPress is to provide a web-service for the aggregation of print orders, offering users the benefit of economies of scale achieved through collaborative purchasing. Our primary aim will be the dissemination of open educational resources (OERs). A cost-effective printing solution will add significant momentum to the adoption of OERs as well as incentivise the creation or release of further OERs and allow real classroom use which is a necessity for the iterative, collaborative development cycle of quality OERs.

OpenPress is beginning to take some real shape and we’ll start spreading the word about our first phase in the next 48 hours. Before we roll out the online service we will do a pilot run. This will be a full manual implementation of the process as a proof-of-concept. The second phase will be the development of the web-service that implements the process, informed by our learning from the manual process.

For the manual process we have selected the FHSST Grade 10 Mathematics book. The FHSST books and other open education resources have no authors’, editors’ or publishers’ royalties so we only need to worry about the cost of printing. Everybody knows the more of something you produce the cheaper it gets per unit and the same holds for printing. If we all put our individual orders together, everybody benefits from a better price. Our goal is to help everybody get the cheapest possible price by finding as many orders as possible and aggregating them together. To make it even more attractive we’ll place an order for the first 1000 books (R50,000) which we’ll donate to severely disadvantaged schools. This will guarantee a maximum price of R50 per book and every additional order will makes the price cheaper for everyone.

Details for the first pilot print run:

The process:

• We place the first order for 1000 Grade 10 FHSST Mathematics books with OpenPress
• All interested schools, organisations and individuals submit the number of books they are interested in to OpenPress at openpress@siyavula.org.za
• Submissions must reach us before the 15^th of November 2009 (Please note that this is merely an expression of interest and not binding. We will not hold you to it in any way)
• OpenPress finds the best possible price for the total order
• OpenPress contacts everyone who expressed an interest with an offer at their best price
• Parties wanting to commit to this print run completes a contract for the number of books they require
• OpenPress prints the books, followed by payment and collection.

Other titles:

We are accepting expressions of interest for any of the 6 FHSST Books, Mathematics Grade 10,11 and 12 as well as Physical Science Grade 10,11 and 12 as part of the manual process. However, we can only guarantee a maximum price of R50 on the Mathematics Grade 10 book to start.

As soon as the OpenPress site goes live we will make available the full list of open titles available for order. To date we have located literally 100s of titles that have been released under an OpenPress-friendly copyright license.

I wrote a blog post a little while ago about crowdsourcing the OpenPress logo. That was before we’d actually finished the process and I just wanted to take a quick moment to reflect. This post is long overdue so I’ll keep it short and just hit the highlights. For the record, you’ll find a ton of people for and a ton of people against crowdsourcing, if you want a logo, its an option and the better you manage the process the better your result will be (paying more also helps).

The experience of crowdsourcing the logo on 99Designs was interesting, fun and ultimately quite exhausting. For the record, we are happy with the logo that we have and, as we’re moving forward with the project, here is the logo for your viewing pleasure:

As with anything design-related there are always going to be opinions so it will be interesting to hear what other people think of the logo. We had a total of 140 designers submit 840 different designs. Some extremely good, some incredibly bad. We prepayed a prize of $500 and, as soon as we had a number of logos we felt we could live with we, guaranteed the prize.

Prepaying and guaranteeing a prize are incentives for designers to participate in your competition. We felt that guaranteeing the prize about half-way through the competition would be a strong, positive indication that the designs we had not yet eliminated were serious candidates.

The fact that design is very subjective is a key thing to bear in mind when using a crowdsourcing tool like 99Designs. That is not to say there are aren’t some well established guidelines for design. We found this list of 45 Rules of Great Logo Design to contain a number of good basic checks even though we broke a few. I’d recommend looking through that list (or any list for that matter as there are many) and then making as specific a specification for your logo as possible before launching your competition.

To get the most out of 99Designs do:

• realise that not every designer who will submit a design is really a designer (an amateur with MSPaint can still sign up)
• specify your constraints very clearly including things you’re not sure about (they need to know what parameters they have to play with)
• provide regular feedback (to guarantee convergence)
• be consistent in your treatment of logos
• update your specification early on if you want to stop trends developing
• eliminate designs that you don’t like in a timely manner
• make a checklist of basic things and stick to them, things like:
  • must work in B&W
  • must scale well
  • etc.
• prepay your prize

and definitely don’t:

• don’t spend the first day refreshing your browser, we were really excited by the first 20 designs discussing them all in detail but when you’re going to get 840 its not worth getting too bogged down early
• get involved in any offline discussions with designers about their designs
• consider designs not submitted through the 99Designs site
• try to provide a detailed response to everyone
• get too excited in the first day

If you provide regular feedback that is consistent and you make it available to all the designers by keeping it in the competition you are more likely to get something you like as well as keep the designers interested and keep frustration levels down. If you oscillate and are inconsistent designers will get irritated and move on. The designers definitely feed off each other and the whole contest becomes a dynamic system you get to prod and with many designers tweaking/riffing off each other tempers can flare-up. As long as you’re consistent and following the rules it should converge.

If you do embark on a crowdsourcing experience brace yourself for a busy week and have a look at this post on bad logos to keep your humour up.