The idea of testing crowdsourcing against a more traditional design process came up last year (you can read about it here) and we decided to do an experiment as we needed new covers for the two sets of Free High School Science Texts (FHSST) textbooks.

The crowdsourced covers from 99Designs came in first (featured in this blog post):

Here are the full sets of covers for you to compare (you can leave your opinions about which win in the comments if you like):

Grade 10

Grade 11

Grade 12

Results

To be honest, I prefer the crowdsourced Physical Science covers but, in the interests of full disclosure, I managed that process. I am quite happy with the Mathematics covers. If we had had different designers as part of either process the results could have been different so this is certainly not a definite test with a control group but a little subjectivity makes for better coffee-time debate.

Obviously, the key thing to an end product you can live with is how you manage the process no matter which route you go. I have discovered I am very bad at working with an individual designer on a single project. I find it difficult to articulate what I want and what it is that I like and don’t like. These things make me a difficult client, although I feel this is mitigated slightly by the fact that I’m aware of it.

99Designs competitions are more tolerant of my choices and I feel that some designers seem to be able to figure wear I’m going based on what I rate designs and what designs I eliminate. I’ve started to standardise my own process for running competitions on 99Designs.

Competition

• The competition benefits from designers competing and feeding off each other – if you run a blind contest (where designers cannot see each others entries) you’ll lose out on the complex feedback between different design ideas and will basically be running about 30 individual design processes, requiring a lot more feedback and time. If you’re a bad client (like myself) that struggles to give good feedback this will really be difficult.
• For the first 3 days don’t give a rating higher than 3 stars – no matter how much you like the design. Immediately after launching the competition make a comment to this effect otherwise designers will get unhappy. They’ll grumble but live with it if they know it is your policy. If you don’t comment and do it they’ll just think you’re being difficult. You may well find something you really think you like in the first 48 hours but those strong, early contenders have always faded out as the feedback starts to take place.
• Provide feedback often – everyday – the competition only lasts 7 days but will take a couple of hours a day. If you don’t put effort into getting a design you like why should the designers – they don’t have to live with it.
• Prepay the contest – then 99Designs holds the money – you can still get it back but many designers won’t even consider participating if the competition isn’t prepaid. If you are serious about getting a design and have a budget then prove it.
• Once there are a couple of days left AND some designs you could live with – guarantee the contest – this way you can’t get the money back and must choose a winner – but its an added incentive to the designers to get involved. Many of the best seem to wait until the last 48 hours before entering.
• If designs conflict with the brief – eliminate them – and comment to that effect. Don’t let them hang around. If you hate it – eliminate! The competition feeds on what doesn’t get eliminated and on what gets rated highly.
• Give designers you rate individual feedback on their design – not in the general comments section. Don’t pick an individual designers idea and ask everyone to work on something like that.
• Put general feed back or changes to brief in general comments section.
• Don’t be put out if they moan a bit about your poor design choices, it is your design content after all. If they mail you privately just say that you are following the rules and they should go through the competition page – do NOT argue with them about design, especially not privately. Keep it on the competition page.

Brief (this is for a logo design competition)

Give yourself reasons to eliminate the rubbish easily by including things like:

• no drop shadows
• no gradients
• no 3D effects
• no stock images
• no proprietary fonts (otherwise you have to purchase)
• at most 2 colours (maybe 3 ;)
• logo must scale down
• logo must work in B&W
• should be balanced
• colour logo must be presented on white background (if they start
  using textured backgrounds to make the logo look good it is hard to
  compare etc.)

Thats all before you even say anything about what you are looking for. You can easily find much more comprehensive lists for rules for logo design competitions with a quick search online.

Then tell them what you want and include:

• project elevator pitch and link to site or descriptive material if
  you have.

• icon and/or word together
• ideas of colours if you have any
• whether icon should stand on its own
• ideas of fonts

Make sure you ask for:

• Colour version
• B&W version
• Vector files (I’d ask for .ai and .svg files)

I think that crowdsourcing is also a lot more fun than dealing with a single designer and you really do see a lot of different concepts in a very short space of time. I’m sure there is an argument to be made that a longer process might lead a more mature result but I’m sure a counter-argument exists built on intensity and focus for the person running the competition. Ultimately, as with everything in life, you have to make your own decisions and live with them. Luckily taking a chance on a logo for $500 isn’t very risky and worth a shot. The more you pay the more attention you get but for $500 for a logo design I am barely able to keep up with my day job and the competition.

As part of the launch of OpenPress, the FHSST Grade 10 Mathematics book is getting printed. We felt that we needed a fresh set of covers for the books. They will be printed with a colour cover and we felt that the all the Maths covers should have a theme; similarly for the Physical Science books.

The covers will need to be available in the next 10 days or so and we were discussing the best way to get them done. The obvious way to go is to commission the work from a designer and go with that. We also considered the option of crowdsourcing the covers and have had a lot of debate about the merits of crowdsourcing.

To resolve the issue, once and for all (maybe), we’ve decided to run our own internal competition. The prize will be bragging rights as the optimal process for future covers for OpenPress / FHSST texts. We are going to allocate the same budget to the Maths and Physical Science covers, but we are going to have the Maths set commissioned by a regular design firm and we’ll get the Physical Science ones created on 99Designs.

Although it isn’t really a competition it will be fun to see the results. The only way to keep the results comparable is to give both processes the same:

• budget,
• brief,
• quantity and quality of feedback, and
• the same timeline.

I’ll post the brief a little later and the results as well so you can decide which you think is the best set and whether we’ve chosen the best process based on this little experiment. If nothing else this should be fun.

I wrote a blog post a little while ago about crowdsourcing the OpenPress logo. That was before we’d actually finished the process and I just wanted to take a quick moment to reflect. This post is long overdue so I’ll keep it short and just hit the highlights. For the record, you’ll find a ton of people for and a ton of people against crowdsourcing, if you want a logo, its an option and the better you manage the process the better your result will be (paying more also helps).

The experience of crowdsourcing the logo on 99Designs was interesting, fun and ultimately quite exhausting. For the record, we are happy with the logo that we have and, as we’re moving forward with the project, here is the logo for your viewing pleasure:

As with anything design-related there are always going to be opinions so it will be interesting to hear what other people think of the logo. We had a total of 140 designers submit 840 different designs. Some extremely good, some incredibly bad. We prepayed a prize of $500 and, as soon as we had a number of logos we felt we could live with we, guaranteed the prize.

Prepaying and guaranteeing a prize are incentives for designers to participate in your competition. We felt that guaranteeing the prize about half-way through the competition would be a strong, positive indication that the designs we had not yet eliminated were serious candidates.

The fact that design is very subjective is a key thing to bear in mind when using a crowdsourcing tool like 99Designs. That is not to say there are aren’t some well established guidelines for design. We found this list of 45 Rules of Great Logo Design to contain a number of good basic checks even though we broke a few. I’d recommend looking through that list (or any list for that matter as there are many) and then making as specific a specification for your logo as possible before launching your competition.

To get the most out of 99Designs do:

• realise that not every designer who will submit a design is really a designer (an amateur with MSPaint can still sign up)
• specify your constraints very clearly including things you’re not sure about (they need to know what parameters they have to play with)
• provide regular feedback (to guarantee convergence)
• be consistent in your treatment of logos
• update your specification early on if you want to stop trends developing
• eliminate designs that you don’t like in a timely manner
• make a checklist of basic things and stick to them, things like:
  • must work in B&W
  • must scale well
  • etc.
• prepay your prize

and definitely don’t:

• don’t spend the first day refreshing your browser, we were really excited by the first 20 designs discussing them all in detail but when you’re going to get 840 its not worth getting too bogged down early
• get involved in any offline discussions with designers about their designs
• consider designs not submitted through the 99Designs site
• try to provide a detailed response to everyone
• get too excited in the first day

If you provide regular feedback that is consistent and you make it available to all the designers by keeping it in the competition you are more likely to get something you like as well as keep the designers interested and keep frustration levels down. If you oscillate and are inconsistent designers will get irritated and move on. The designers definitely feed off each other and the whole contest becomes a dynamic system you get to prod and with many designers tweaking/riffing off each other tempers can flare-up. As long as you’re consistent and following the rules it should converge.

If you do embark on a crowdsourcing experience brace yourself for a busy week and have a look at this post on bad logos to keep your humour up.