To Crowdsourcing Friends, Foes & Fanatics: Just How Loyal Is Your Community?

Tuesday May 25, 2010 by Matt Johnston - Vice President of Marketing and Community, uTest

Depending upon who you ask, crowdsourcing is either evil, revolutionary, or a next gen of internships.

But one thing that ALL crowdsourcing companies like to preach is how loyal and trustworthy and professional their community is. I know because I’ve read it in 100 different sites. Hell, I’ve written it a 100 different times here at uTest. So why do crowdsourcing companies insist upon telling the world how loyal and earnest their community is?  Maybe it’s to assuage the fears of prospective customers about entrusting their logo design, app development, content production or marketing to a community of strangers. Maybe it’s because if marketers say it enough times, we hope it’ll come true.

The more pessimistic view is that people — cloaked in the anonymity of the web — often act in greedy, selfish, mean-spirited ways (this perspective didn’t make it into the crowdsourcing brochure, by the way). Such dark behavior is well-documented and takes the form of flame wars on message boards, bullying via social media and online fraud.

So which is it — are people good-natured and honest?  Or are they money-hungry malcontents who will do anything to get ahead, as long as they don’t get caught?  Obviously it depends on the people, but I learned the truth about our community this week — and it was a lesson we learned the hard way.

A little background:  At uTest, we pay our testers twice per month via PayPal or Payoneer. And at this point in our growth, each pay cycle involves a non-trivial amount of cash — pretty deep into the five-figure range.  Now, it’s not easy or flattering to admit this, but in our most recent pay cycle, we experienced a glitch that caused us to pay our testers twice. That’s right folks, it was double payday here at uTest!

We discovered the error moments later when several testers emailed us to point it out to us. We immediately split our efforts up into two camps:  one group trying to identify the issue and make sure it never happened again; the other trying to figure out how to get our money back.  I was in the second group.

After exhausting our options with our payment vendors (who were responsive and sympathetic, but unable to help), we came to the realization that we would have to simply explain to our testers what happened and ask them to refund the duplicated payments via a not-so-simple five-step process. Now, I love our community and I think the world of them, but let’s just say I was skeptical. I mean, c’mon, it’s our mistake and it’s free money.

So what happened?

• At 2am on Sunday morning, we sent out an email to all the affected testers… and hoped for the best
• Within 24 hours, more than two-thirds of the money had been refunded
• Within 72 hours, more than 90% was refunded
• And within 96 hours, more than 98% had made it’s way home
• I expect that within a week, less than 0.5% will remain outstanding
• We’re not talking about a few dollars here and there… one U.S. tester refunded nearly $2,000. Several others from the U.S., Canada and Western Europe issued refunds in the $1,500 range. We had testers in developing nations who refunded nearly $1,000 — in places where $1,000 is more than one month’s salary for a QA engineer.

So in addition to proactively bringing this gaffe to our attention, our community went out of its way to refund this money promptly and in full. Sure, I’m still pissed and embarrassed that it happened. But it did, and the takeaways have been surprising and worthwhile:

1. Communities beat crowds. Crowdsourcing companies should stop trying to build crowds or mobs. Instead, invest in building a community. We pour a lot of time, energy and money into our community. From our tester forums to free training programs to Bug Battle competitions to town hall meetings to our uTester of the year awards.  And we don’t do it because it’s always fun or easy or even public. We do it because we believe that communities are better than crowds or unruly mobs.
2. Surprise, surprise, surprise. Even after nearly 7 years of building community-driven businesses, I can still be genuinely surprised by the loyalty and integrity of a large group of people who have never met one another in person.  I wonder how else our community would surprise me if I keep a more open mind.
3. Deep, trusting relationships are a necessity.  We treat the men and women in our community like respected professionals. We work hard to give them a lot of different ways to engage with us on their own terms. And they have always treated us well in return. But when push came to shove and there were real dollars on the line, the true impact of the relationships between us and our community became abundantly clear.
4. My new favorite story. The next time a prospective customer asks me “who are these 25,000 testers and how do I know I can trust them with my super-secret app?”, I know exactly what story I’ll tell them in response.

Now, my fellow crowdsourcing enthusiasts (on both sides of the aisle), let me ask you this:  when push comes to shove, just how loyal is your community?  While I don’t recommend this particular path as a way of proving it, I would love to hear other stories about what happens when a community is transparent and accountable to one another. Or, if you have them, tell us your nightmarish tales of community deception and betrayal.

Matt Johnston is the VP of Marketing & Community at uTest.  This blog post was originally published on May 20, 2010.  You can find this blog post, as well as additional content on the uTest blog located here.

< back to all blog entries