I am a big fan this whole “minimum viable product” philosophy. I’ve probably adapted it to my own interpretation, but basically I look at it as bootstrapping on crack, get your product out with the minimum feasible offering to start seeing traction. I am trying to be as resourceful as possible with this startup. Using existing technologies, free services, API’s etc. to help build the service and get launched as quickly as possible.
The first place I started was when I had my ‘epiphany’ (that’s what I call the latest idea). I knew that I needed to find out if this was even a viable idea. Asking friends and family is great, but let’s face it, they already love you, they’re your friends/family.
So I decided my step was to get a survey up about the idea and find out what people think, (I think this qualifies as a first step in the MVP Philosophy).
So I did just that, with Amazon’s Mechanical Turk system. I spent $27.50 and surveyed 200 people. There is NO segmentation in this system so I got feedback from anyone, which is exactly what I wanted. (well I wanted feedback from people that have access to the internet since this is a web startup).
I wanted to see how many users from the “general” population that has access to the web would find this service useful. I also wanted to see how this broke down by Age/Gender as well as ways in which they could see using the service.
Here’s what I asked in my survey:
I gave a brief description of what the website/service offering would be then asked the users:
- Gender
- Age
- Would they use the service as described above
- Give me 3 examples of how they would use the service.
- General feedback or ideas on the service, why they would or would not use it.
The information I got back for my $27.50 was INVALUABLE. I found from that 1 survey, how to basically build my product for launch. What features I had to have based on how users would use the service. I also realized I could basically cut my current feature set in 1/2 because what I thought people would want, wasn’t even mentioned.
My Results:
- I found out that 73% of the people surveyed would use my service (that validated the idea for me).
- Then found that of the 27% that wouldn’t use the service, WHY they wouldn’t use the service, then I added those common “no” reasons into my MVP product for launch, in hopes that I could convert even more.
- I also found out to my surprise that more men than women said they’d use the service. I fully expected it to be the other way around.
It’s pretty easy to create a survey using the Turk System. But I highly recommend doing some surveys or trying it out with small “batches” first so that you can figure out how to use the system before processing 200 questions.
Bottom line is, this was a great way to just throw out the idea, get validation and see what people had to say. This also helped fine tune my spec document for product build based on actual feedback on the product idea/concept.
<SelfPromotion> You can sign up at http://www.swayable.com to be notified when I launch, or “like” the Swayable Facebook page. You can also follow me on Twitter </SelfPromotion>
harperlindsey.wordpress.com 
Pingback: Tweets that mention How I Used Amazon’s Mechanical Turk to Validate my Startup Idea | Ramblings of a Web Startup in Progress . . . -- Topsy.com
I think the article will be a lot more interesting if you mentioned your service, and the reasons people stated they would/wouldnt use the service. But I get it, your in stealth mode right?
Great feedback! I will update the post with more data when I am out of stealth mode 😉
Pingback: Using a Virtual Personal Assistant for Your Startup | Technology and Web 2.0
Pingback: Using a Virtual Personal Assistant for Your Startup
Pingback: kool-gadgets.com » Using a Virtual Personal Assistant for Your Startup
Pingback: Surfer Watch .com - Using a Virtual Personal Assistant for Your Startup
Pingback: Using a Virtual Personal Assistant for Your Startup | Tech News Ninja
Pingback: Using a Virtual Personal Assistant for Your Startup | ECtimes.com
Pingback: Using a Virtual Personal Assistant for Your Startup | 365Online: E-Commerce en Online Marketing
Pingback: Using a Virtual Personal Assistant for Your Startup - www.Korallenkacke.com
Pingback: Using a Virtual Personal Assistant for Your Startup | The best Tutorials
Pingback: Using a Virtual Personal Assistant for Your Startup » Vizio Internet Apps
Pingback: Using a Virtual Personal Assistant for Your Startup | Tech Toinks!
Pingback: Ly Technology » Using a Virtual Personal Assistant for Your Startup
I would love to hear more when your out of stealth mode. I just signed up myself and have taken a few qualification tests. Any other tips you could provide would be very helpful. Thanks again!
Thanks Roy! I am hoping to be out of stealth mode soon. Just wanting to dot all the I’s and cross all my T’s before blabbing off to everyone 🙂 if you have any specific questions in regards to tips, let me know. I just started this blog last week so I am not sure yet what people will be interested in hearing about. I am just writing about my experience/path.
Pingback: ArticleSave :: Uncategorized :: Using a Virtual Personal Assistant for Your Startup
Pingback: Using a Virtual Personal Assistant for Your Startup - 25 Popular Blogs - Popular Bloggers.com
Pingback: Using a Virtual Personal Assistant for Your Startup | nyutrendy.com || News and Trends In The World
Pingback: Using a Virtual Personal Assistant for Your Startup | IT News Wire
Pingback: IBM Helps Tennis Fans “See Through Walls” with Augmented Reality
I know you said it was unsegmented, but do you have *any* indication about the nature of these users? Are they super geeky, or just normal people?
Great question Mike – Unfortunately no, the only information I captured with regards to demographics was Age/Gender. This worked ok for my startup as it’s a consumer facing/free service. It definitely won’t work for all as you can’t really segment the audience that answers your questions in advance on the Turk System. Although you may be able to do this in other services like Crowdflower.
The problem is that the “workers” on AMT are not your typical web user and for many startups very different than their target customer. Be careful on how you interpret such biased data.
Thanks for the feedback. I am definitely not basing my full startup on a survey of 200 people, but it did validate the idea in the sense that I got great feedback. (Not meaning just the yes/no answers). The text feedback which was 3 of my 5 questions was the most valuable, and I was able to update several features that I thought I needed and found that user’s comments were showing me to go in a bit different direction. But yes, you are correct, you have to look at any survey results through they eye of where you got them. 😉
It sounds like the kernel of an awesome idea, but it doesn’t make clear who took the test. Are the Turkers your potential customers, or were they used to get the surveys to your potential customers? As far as I can tell the Turkers are people doing piecemeal digital work for very small sums.
Great question – I looked at the Turkers as potential users of my service, so the survey was directed at them. My startup is a consumer facing free service, so I needed to get some feedback from computer/web savvy users. There are flaws with any survey that you do, and the Turk system for surveying your startup may not work if you are focused on a clear niche, because you can’t segment your audience. For my startup it worked great, I wasn’t as interested in the yes/no answers as I was in the verbatim/text feedback. That is where I found the most value. No system is foolproof though, and I took that into consideration when doing this survey.
There’s an incredible selection bias involved- those who use Mechanical Turk are clearly computer savvy, have a certain amount of free time, and are looking for spare money on the internet. Please (please) do not consider Turk respondents to be indicative of the general population.
You are correct, the Mechanical Turk system is not necessarily indicative of the “general” population. What I find interesting is that many people will pay for Google Adwords to run the same survey and then offer some kind of incentive to complete their survey. My question is how is this different (the users are still incented to complete the survey)? I understand if you have a niche focused product, than Google Adwords/survey combination would be much preferred to Mechanical Turk because you can segment by keywords etc. But for my startup, where it is a consumer facing, free service and needs to be used by people that are web savvy/computer savvy. So I determined the Turk system would give me the generalized feedback I needed. I was more interested in the 3 of 5 questions that had text fields as those were literal responses/feedback that I found the most valuable.
Sure they would use it, but would they pay for it (at your price point)? I would guess that when you phrase it in terms of “whether you would pay to use this service,” you get radically different numbers and qualitative feedback. But using Mechanical Turk is a brilliant method.
Agree with your comment in regards to getting radically different feedback if I asked users if they would pay for this service. The startup I am working on (in stealth mode, which is a completely different conversation) is a consumer facing web application and iPhone application that is free for the user so I didn’t have to ask if they would pay for it. However, that said, it may be interesting to just do a survey to see if any users would pay for it 😉
As the previous comments I think you should not be 100% confident on the Turk data. Maybe you should post some limited functionality for normal people to write their thoughts about it.
Regards!
I’ll be posting more about http://www.swayable.com shortly and looking for more feedback. I definitely am not relying 100% on that data, it just validated the idea, and gave me great feedback (the text feedback). Thanks for your comment.
Pingback: To be Stealth or not to be Stealth with your Startup | Ramblings of a Web Startup in Progress . . .
Hey Lindsey,
No opinion on the relative selection bias expressed by the previous commenters. Only you can say whether their feedback is helpful.
My issue is actually with the quality of the test, regardless of who you ask. The problem is that it’s been amply demonstrated in studies that if you ask people if they’d use a service or purchase a product, they will often say yes if they think you want to hear yes. There is a significant primal desire to tell people what they want to hear. That’s why finding mentors who aren’t afraid to burst your bubble is so important.
The real test is that if you immediately follow up with these folks who said they would give you money and actually ask them to commit to being a real customer. Say “In that case, would you be willing to become my first customer? If you pay right now in advance, I’ll give you the first year for 60% off!”
You will be genuinely alarmed at how quickly those yes answers become no answers.
A good measure is whether you can easily find 10 people that will happily give you real money to get your product as quickly as possible.
You’ll have a hard time convincing an investor to get on board if you can’t find these 10 people.
Agree wholeheartedly with Alvan Tan’s comment: asking survey respondents and potential customers whether they would PAY for something is hugely different, and important.
Pingback: Using a Virtual Personal Assistant for Your Startup | Bina-web.com - Web Tips
Pingback: Startup Day #1 « The computer only does what you tell it to do.
Lindsey, you inspired me to run a smilar test. Results are here:
http://bittyblog.posterous.com/using-mechanical-turk-to-confirm-we-are-on-th
Thanks for the inspiration.
Awesome Scott ! Just read your post and looks like you got some great feedback 😉
This is a wonderful post and may be one that needs to be followed up to see how things go
A partner emailed this link the other day and I’m eagerly awaiting your next write. Proceed on the world-class work.
Pingback: Josh Mather » Bookmarks for September 1st through September 20th
Pingback: 5 Lessons Learned: User Testing with Amazon’s Mechanical Turk. |
Pingback: Crowdsource Your Market Research in Canada | Two Birds, One Stone
Lindsey,
thats some $27.50 you spent. It has already produced so much more for you. I have a simple question – the 200 odd respondents that you surveyed, were they provided by Amazon or were they from your own list of email ids? It would be a big help if you could shed some light on this.
Hi, the people were all provided via Amazon’s Mechanical Turk system. All I had to do was create the survey 😉
Wow! This really is 1 of the most beneficial blogs I’ve ever occur across on this subject. Merely Amazing
http://harperlindsey.com/2010/09/01/how-i-used-amazons-mechanical-turk-to-validate-my-startup-idea is the best! you have many fans in my country.
You have a really useful blogging site, I’ve been here reading for about one hour. I am a newbie and your accomplishment is extremely inspirering to me.
I don’t know what to talk about except that I have enjoyed reading! Excellent overall blog site.
I’m still learning from you, while I’m improving myself. I certainly enjoy reading everything that is written on your site.Keep the tips coming. I enjoyed it
Pingback: How not to test your startup idea; how to test your startup idea. | wademeredith.com
About how long did it take for your results to begin coming in? Did you have to wait several days? Did they start coming in shortly after you created your HIT?
Hi Ryan,
Results started coming in within 10 minutes of posting the HIT to Mturk 😉
Pingback: Quora
Hi. I was just doing some random Googling, and I came across your page. I actually participated in some of your HITs on mturk, I believe. I can tell you a little bit more about the assortment of people turking. I, for one, am your typical web user. Most of us turking seem to be just that — depending on location, I would say. There’s a large forum of Mechanical Turk users (well, the most dedicated of us) at Turker Nation if you want to get a better understanding of what (and who) mturk actually is. Requesters are encouraged to create an account and ask questions, post information about their HITs, etc.
It’s not always easy to find a good group of solid workers on mturk, but you can customize who can and can not do your HITs with mturk’s built in qualifications — most importantly, location. You can open your HITs up to U.S. only (which I hate, being Canadian), or you can exclude several countries which have proven to be problematic for many requesters as far as quality work goes. You can also set up various qualification tests to try and customize your worker pool.
I would have to agree that a lot of people taking these surveys would answer positively whether they actually felt that way or not. If I were setting up a survey HIT such as this, I would make it VERY clear in the instructions that you want honest feedback — positive or negative. Let workers know that there will be no negative repercussions for honest, negative feedback. I think if that was stressed, turkers would be more willing to offer up honest feedback without fear of having their work rejected.
Also, I wouldn’t recommend pricing your HITs too low. A short survey for a dime is fine, but if your HIT is any more involved than that, you won’t get dedicated, quality workers unless you pay a fair wage. Since a lot of turkers are looking to make something close to minimum wage, you have to think about how long your HIT is going to take them, and pay accordingly. Some workers will work on any HITs, but those aren’t the workers you want.
Lindsey,
Thanks for the post. I had been milling over it for awhile and finally ended up giving it a go and compared the end results to some other market research surveys. I wound up getting extremely useful information using mturk.
http://kurtgrandis.com/blog/2011/02/01/surveying-mechanical-turk-to-validate-a-startup-idea/