Lindsey Harper posted an interesting article about using Amazon Mechanical Turk to test her start-up hypothesis.
Many of the comments were not in favour of the idea, because of the [perceived?] inherent selection bias of asking Turkers for their opinion. Only she can say whether their feedback is helpful; she’s in stealth mode so it’s hard to say for sure without knowing her idea.
My issue is actually with the quality of the test, regardless of who she asked. The problem is that it’s been amply demonstrated in studies that if you ask people if they’d use a service or purchase a product, they will often say yes if they think you want to hear yes. There is a significant primal desire to tell people what they want to hear. That’s why finding mentors who aren’t afraid to burst your bubble is so important. I strongly suggest that you consider reading Influence: The Psychology of Persuasion if you’re interested in understanding why people do the things they do. It’s a fascinating book.
If Lindsey really wants to test her idea’s sales potential, she needs to immediately follow up someone who says they will purchase a product with a request for them to purchase your product. Customers pay you money, so they need to open their wallets in order to demonstrate being a real customer. Say “In that case, would you be willing to become my first customer? If you pay right now in advance, I’ll give you the first year for 60% off!”
Many founders are genuinely alarmed at how quickly those YES answers become awkward NO answers. Apparently, your sure sales aren’t so sure about why they need your product. Perhaps you’re not making them happy or getting them laid?
A good measure is whether you can easily find 10 people that will happily give you real money to get your product as quickly as possible. Let’s face it: you’ll have a hard time convincing an investor to get on board if you can’t find just 10 people. This fellow thought it would take 6 months to get 200 customers. It actually took 30 months!
TL;DR: Founders need to be able to get 10 people to pay them in advance, or their market doesn’t exist. You shouldn’t worry about scaling your technology stack to millions when it might take 30 months to get to hundreds. Your sales process always takes way longer than you could ever expect. And perhaps most importantly: