How can you determine if something is worth building? Recently at the Warm Gun Conference Instagram founder Mike Krieger talked about what they called “The Wizard Of Oz Techniques For Social Prototyping” – what I’ve heard called “404 Testing” – where he said:
Krieger says him and Systrom tested an early version of a feature which would notify you when friends joined the service. Instead of building it out, they manually sent people notifications “like a human bot” saying ‘your friend has joined.’ It turned out not to be useful. “We wrote zero lines of Python, so we had zero lines to throw away.”
What could be a better way to find out if something was truly valuable to your customers – then to fake it. There are a couple stories, one dating back to early computing days when a company wanted to see if they could get admins to use voice dictation software. At the time, this was a very expensive project, with huge technology hurdles. Before starting to invest in this project, they decided to run a test. They installed a “working” prototype into an office, only it wasn’t real. While the admins talked, someone was listening and typing back what they said.
After some testing they found that the admins didn’t like the product, that it was taking away from their tasks, and they preferred other methods for input. This would have been a multi-million dollar project, that most likely would have flopped. They spent almost nothing to find out this product had very little value to their target market.
I’ve thought about how this could work for smaller feature sets – and then came across this:
It’s from a new travel site called: http://www.mrarlo.com/
While at first you could be thinking “lame, why not just add a under construction gif to the page?” – but if they’re tracking the clicks for this – they’ll quickly determine what % of visitors would be interested in the feature, and is it worth the investment to build out.
When asking customers if they would like feature A or B, they’ll usually say “how about both?” – to them features are free. They don’t know the development that goes into something as seemly simple as “neighborhood” for instance.
404 testing is a great method for finding value, from a single feature to an entire product. Remember, development is expensive and will keep you from delivering features your customers really want.