May 26th 2012 | The Economist published a very interesting article titled “The roar of the crowd – Crowdsourcing is transforming the science of psychology” that discusses how the psychology community is turning to crowdsoucring for better test sample populations.
There is growing evidence that supports results of studies conducted on undergrad student is widely different from the result conducted on more diverse population. Can the same be true for usability testing? Well if you are developing a digital application for say, stock market analysis and your test population was “A” – financial professionals and “B” – global population of mixed professionals, you’d get different results. In this case crowdsourcing user testing would not be that enlightening because you’re developing a product for a very targeted audience.
However, if you were developing a product for the general public crowdsourcing user testing does offer some benefits. Although one can argue that user testing for digital devices is another very small targeted market, when compared to the global population, most usability testing does center around human factors.
While Elance, arguably the most popular crowdsoucring site, has championed crowdsoucring for the IT community a new scientific crowdsoucring site has emerged called Mechanical Turk from Amazon. With over 500,000 people (40% American) Mechanical Turk is a new source for psychological testing for academia. Whether Mechanical Turk can be a new asset for usability test is not clear yet, but with the benefits that crowdsourcing encompasses it is hard to imagine it wouldn’t find some traction in the usability space.