Dan:
Hey. Hello everybody, it’s the Lighthouse London podcast. I’m Dan.
Tom:
And I’m Tom.
Dan:
And today we are talking about how to get meaningful feedback from a prototype for “It’s his prototype month”.
Tom:
It certainly is.
Dan:
…here at Lighthouse.
Tom:
It is.
Dan:
We are prototyping everything from new footwear to new ways of typing on a keyboard.
Tom:
Oh really? How’s that one going?
Dan:
It’s a disaster.
Tom:
Getting any meaningful feedback from that? Don’t do this!
Dan:
Not everything is taking twice as long but we are learning so much. No, we are just having a bit of a chat about prototyping in general with the blog is going to be full of prototyping goodness.
Tom:
Well, last week, we just put up our video about the Prototype Sprint, which is a little service that we as Lighthouse offer, wearelighthouse.com/blog if you want to go and check that out.
Dan:
Absolutely. It’s come out very well.
Tom:
Its very lovely. Very nice, I think you look quite dashing in it.
Dan:
And quite gray. Edit that, edit that. Thanks to Threefold Films, our friends there. Tom and Graham did a sterling job.
Tom:
Quality guys.
Dan:
Absolutely.
Tom:
Even brought us a steady cam which is quite exciting.
Dan:
Yeah, that was amazing. I felt like I was in a proper film.
Tom:
Yeah.
Dan:
Enough banter, let’s get on. Never enough banter we all know that, but let’s get on with chatting about prototyping.
Tom:
Well, yeah, so this assumes that you’ve come to Lighthouse London, we’ve done a prototype, we’ve put something together for you. It’s amazing…
Dan:
Well, or you’ve made your own.
Tom:
…And you’ve now got a prototype that you want to… wait that’s not to say they are doing nothing, what’s it for?
Dan:
Well, it’s for testing…
Tom:
And learning…
Dan:
And learning. In that order.
Tom:
That’s what I was fishing for there.
Dan:
Testing and learning. And what do you need to test and learn?
Tom:
A prototype?
Dan:
No, customers.
Tom:
Customers! Damn.
Dan:
Deary me. What we are going to chat about today that is that bit that we don’t do for people.
Tom:
Well, we’ve done it sometimes.
Dan:
Well I suppose. But that’s usually down to the entrepreneur, founder, business owner, or whatever you want to call them.
Tom:
I think it’s right that’s it’s down to them really because, you know, we can show people how to test with an end user but ultimately if you’re going to run a business that relies on customers, if you’re at all being innovative or trying something new out you’ve got to get good at learning from customers, so, making the prototype is just the first half of it. Using the prototype to get feedback it’s the…
Dan:
Well it’s the main thing it’s almost more important because you know you can use a prototype of vary different qualities and if you can get a lot of feedback out of not a lot of prototype, then you win.
Tom:
Well and it’s all about you getting inside your customers head, isn’t it? So, why have someone else doing that when you’re the one that needs to learn what your customer wants, so you’ve got to be the one out there talking to that person.
Dan:
Agreed. I think the key things to talk about here are who you should be testing it with, what sort of… what does the test look like, what types of questions you should be asking, and then what do you do with all that delicious feedback.
Tom:
Yum, yum, yum.
Dan:
So first off, who should you be testing it with?
Tom:
I’ll tell you what, who should you not be testing it with.
Dan:
Yeah, okay. You should not be testing it with anyone who’s going to be biased.
Tom:
Yep and that means your mum and dad, your mates, people who are kind of supporting you on your business journey, anyone who’s an acquaintance really.
Dan:
Unless those people are customers, but even then, I’d say someone that was compromised that much would not
Tom:
There would always be a level of bias there, but this needs to be ideally or many of of these people should be strangers.
Dan:
Yes, completely. They should people who really don’t care about you or your business but have the problem that your business is trying to solve.
Tom:
Definitely. Where do you get these people? Where do you find them?
Dan:
Well, you’ve got to get creative. I mean, you’re going to have to find customers eventually. So, it’s the same rules apply. You know, if you’re thinking of… if its a sort of a niche thing and there’s forums online then that wouldn’t be a bad place. Somewhere where people hang out physically is better.
Tom:
Yeah, if I was making something for a sporting start-up I would go to the gym or wherever, or the sports where people are playing there and I would just grab some people and ask them some questions.
Dan:
It does take a little bit of bravery, I think.
Tom:
Oh, totally. I think its not easy to do that. Its quite hard to walk out in the street and describe someone a, say… whip out your phone and start showing them a prototype but I guess its something you get good at once you’ve done it or get less embarrassed by it once you’ve done it a couple of times.
Dan:
Absolutely. I’ll tell you what not to do. Very similar scenarios to my GCSE geography project where I had to go…
Tom:
I know it was a disaster.
Dan:
…well, wait. Where I had to go and ask people about what they thought about the split car park in Beckett’s Safeway which dates it because that’s now a little… and yeah. I’m a fairly confident person, don’t mind talking to people, but I went out on the high street with that questionnaire and just did not want to do it. It’s a different type of confidence required. So I went and sat and made up all the answers. So actually, I proved my hypothesis pretty nicely, thank you. Got not too bad a mark in that little bad boy.
Tom:
I’m going to let the school know immediately and your result should be stricken from the records.
Dan:
I think it showed initiative. But yeah, you don’t want to do that, you can’t be lying to yourself about whether or not people like your product. You need to find customers, incentivize them, you don’t need a lot either that’s the thing. I think…
Tom:
Yeah, you need to ask a few but I think people would be surprised that you don’t need to ask hundreds of people that stuff. You need to get a good section of people.
Dan:
Yeah, classic 80/20 rule. You learn 80% of the problems from talking to 20% of the people. So, I would suggest if you can get 5, that’s a start.
Tom:
That’s a good start. Yeah.
Dan:
And Its so infinitely better than zero, that you really will start learning. I mean when we do our prototypes spin that’s the kind of number we are telling people to go out and test it with. So I’ve met my customers, somewhere… what sort of things should I be asking them?
Tom:
I think its important to not pitch to them immediately, so you don’t want to launch it and say this is our idea, we are thinking this and we are thinking that, this will do that, this will do this. You’ve already sort of guessed what their problems are and you’ve hypothesized what the solutions might be and you’ve made a prototype but really you want them to tell you that you’re correct. I’d always recommend that you should ask them a couple of generic questions about this industry, this sector, or whatever it is and just let them talk to start with.
Dan:
So lets imagine there was something like Uber, before Uber existed.
Tom:
Yeah, yeah totally.
Dan:
So you turn up at a taxi rink maybe…
Tom:
And you say, “you having problems getting a cab?” And they would say, “yeah I am.” And you’d say, “well tell me about that.” You don’t then whip out your prototype and say “does this solve your problem?” You ask them to tell you what their problem is and ways they have tried to solve it themselves.
Dan:
Completely.
And when you show the prototype, I think, depending on what the prototype looks like of course, how finessed it is. I think it’s good to stay quite vague. Don’t prompt people too much. You don’t want people forming opinions because they know how this thing was created. If people think it was created in five minutes they are going to think its not too important what they say to you about it and actually you want it to be important, you want it to be as if this was the real thing. So, staying vague about the fact that it’s a prototype, what exactly they are looking at, you really just want to stop and listen to what their response to it is.
Tom:
Totally, yeah.
Dan:
Then dig them out of any holes if they completely don’t get it. That’s something you want to test it, okay, no one understood the first pages of the prototype. Start prompting them but really it’s very similar thing to how you conducted usability session. And obviously that is also something this person probably hasn’t done. But…
Tom:
That’s a life skill that most people know isn’t it?
Dan:
Yeah, I dealing with it all the time and its very, very annoying. Its very similar to that in that you just want to ask open ended questions, the more you can get them talking the better.
Tom:
If you find you’re speaking more than them then something has gone wrong, you know?
Dan:
Completely.
I think there’s some really good books on, or at least little bits of books on how to do this kind of user research. I think we can stick a few in the show notes. The usability one that sticks out is “It’s Not Rocket Science” by Steve Krug. That’s a really good practical guide to how to talk to people and illicit information out of them when they are using something.
So yes, don’t be too prescriptive with your questions. Set a scene, show them it, and then let it run its course. You’re not going to get any extra time people time so make sure that you do find out the things that your big assumptions are.
Tom:
Yeah, you do need to set your goals of what you’re trying to learn for each session or each person and then if they are veering off course wildly to that and you find what they are saying isn’t useful then you’ve got to prompt them to get back on to it without too much help, but you need to have a checklist of what it is you’re trying to get otherwise you’re just flail aimlessly in this conversation.
Dan:
There’s probably a couple of things aren’t there? There’s probably a couple of main points, like, do you have this problem with THAT feature be something…
Tom:
Presumably because there’s stuff in the prototype that you know hasn’t been solved yet so those are the kinds of gray areas that you really need to get to.
Dan:
And what you’re looking for as well is, when you are asking these questions, is just for someone to look at prototype and fill in the spaces with their imagination, because if you have a… often we’ll do this when we are creating our prototypes. You hint at features that you don’t actually build out. Say you have a add button or a kind of, like, more info button next to a certain feature and it doesn’t go anywhere in the prototype but you can ask people, if people notice it, and say oh okay, I click that, what’s behind that? Rather than say anything just say what do you think is behind that? And you learn what in their mind would make sense behind that which in turn is pretty much now know what they want to see there.
Tom:
Completely.
You spend no time designing that page thinking about that flow or what you might have done but having if every user says the same thing you know exactly what you’ve got to build and then you can just go ahead and do it.
Dan:
So, the key points there are: remain vague. Don’t steer them too much. Know what you want to find out though so that if coming near the end they really haven’t gotten anywhere near it at all you can still illicit that last piece of learning out by saying okay this is what it is in this scenario, would this feature be helpful? Or do you think about this problem in a different way because that’s really important to make sure that you do learn as much as possible. But it’s just that I think, half the time you learn the stuff you’re not expecting to learn so that’s why you’ve got to stay open ended on it.
You know the things you, I think a lot of time when we prototype and people say well, what am I going to learn? And the answer to that is well, I don’t really know. That you’re going to learn what you don’t know yet about this thing which is sometimes hard for people to justify doing. But once they’ve done it, those surprise things…
Tom:
That’s what it’s all about isn’t it? You know, you don’t want a prototype to be something highly developed. It needs to be something that has got a lot of questions behind it that you can start asking because at this stage there’s a lot you don’t know and you’ve got to find all that out.
Dan:
If you’re both stuck in testing then you’re process was flawed and its not that there’s nothing to learn about your idea.
Tom:
You’ve asked the wrong questions or you’re not listening or who knows what it could be there. It could be many reasons why that’s happened.
Dan:
Too certain that the idea is going to work. That’s normally what the problem is, I don’t need to test this, its going to work.
Tom:
And sometimes we put stuff in there that we feel is wrong in the prototype because it’s just testing that thing. There’s been very little time gone into actually getting the stuff in front of someone, so, you’ll put something in there that you’re really unsure about and make sure to point it out and people will go, no that’s rubbish. But that’s great feedback. Much better that and finding out it was useless.
Dan:
And so you’ve done this?
Tom:
Um-hum.
Dan:
You’ve got a notepad full of notes, or recordings, or whatever permission they’ve given you to document them. What should you do with that feedback?
Go away and change everything? But in all the features they ask for, and change everything to what all of them said so that they are all happy?
Tom:
That’s what you do yeah, completely. Launch, press the button and go.
Dan:
And then build it…
Tom:
…and done. A million pounds…
Dan:
Absolutely. Well it depends who they were right? If you do think one of them has got a million pounds and they ask for it to be a certain way, then, you know… invoice them and then do that.
But no, generally, I think one thing we see a lot is you will find out the things about your idea aren’t quite what people want or the people want something slightly different. The thing is don’t panic about that. You don’t need to blindly follow this feedback. Its the same as feedback from users across every aspect of product design. You’ve got to treat it more, you can treat them as individual things, it’s more, I think someone gave the metaphor, its more like a kind of river and you kind of get in the river and let it float past you and start understanding which way its flowing, how strong its flowing. The feedback has got to be taken as a whole. It can promote, prompt new ideas but if you’re… people will ask for things that they don’t actually want. You’ve got to scratch below the surface and try to work out why they’ve asked for that.
Tom:
People don’t always know exactly what they want, I suppose, the more people you speak to, and again it doesn’t need to be loads and loads and loads but if they are all saying the same thing you need to take note of that.
And if a couple drop in something then that’s something you should then be exploring later on. Keep it all in mind, keep it in the back of your head and bring it up when you have an opportunity to do so, but don’t just react on everything that everyone says. It’s just a nightmare to do it. You will never keep track of it.
Dan:
Never keep track of it and you’ll never focus. Someone asking for a feature doesn’t, people give feedback for all sorts of reasons and it doesn’t necessarily mean that’s going to then make your product win. Its your job to feed that learning back into the product in a useful way. You’ve got to use your knowledge of the customer and dig below the surface and if you can, it’s not easy to dig below the surface, it’s not always obvious, it’s not an exact science. Perhaps that’s what’s your next round of prototyping will try and prove, whether or not that specific thing should go in or not.
Tom:
Yeah, definitely. That’s the thing, you can go around again on a prototype easily enough. Make some changes to it, that won’t take much time at all. Get out there an ask another five or ten people.
Dan:
I believe the term learning loops was trademarked in a podcast earlier this year.
Tom:
It might have been, yeah.
Dan:
And just using it again there and it’s penny royalties because it was me that trademarked it. So…
And no one else has used it yet as far as I know. If you do hear of anyone using it, please, please let me know.
Tom:
Or maybe we should buy learningloops.com
Dan:
I aren’t look whether or not it’s available because if it isn’t then I might be some counter sued.
Tom:
Wow.
Dan:
I feel that’s pretty much covered how you should go about getting feedback.
Tom:
That scratches the surface for sure but there’s some good advice there. Another book that we should mention is “The Mom Test” which I believe we have talked about  before.
Dan:
I thought it was sexist.
Tom:
It’s not sexist.
Dan:
It turned out not to be…
Tom:
It’s just sexy.
Dan:
Sexy advice.
Tom:
I mean, it is, you need to read that if you’re doing that kind of thing. It’s full of information to do with interviewing people from this kind of stuff and goes into much more detail than we have now to many other hints and tips that you really cannot do without if your going through any type of process at all.
Dan:
We will say it again, another book that we have covered before, covers this stage as well as running lean and valued proposition design as well that these, I think at this point when we are doing these couple of months to starting an ideas, prototyping ideas, I think those books are very influential in how you go about doing that. And really practical. They are not just, kind of like, oh how does an expert do it or my tales from my amazing career or searching getting feedback from prototypes they are actual bullet pointed lists how to do these interviews, where to find people, what the conditions should be. Just practical stuff. They’ve helped us loads.
Tom:
Yeah, The Mom Test one is nice because it admits that you’re not going to be great at this to start with. So it knows that when you first get out the to talk to a stranger in the street you’re going to do a pretty bad job of it. And the guy says I did that myself, so it is realistic to expect that your going to get better over time and there’s some really good advice in those books about where to start and how to start honing your skills.
Dan:
Absolutely. I’m always grabbing them the night before doing anything like this and cramming before pretending to be an expert in these subjects.
Tom:
Just that reminds me of GCSEs. Anyway, links to all of them are in the show notes
Dan:
You can download my GCSE project…
Tom:
We’ll PDF that up. Get that online
Dan:
If you want to know why that split car park led to Safeway downfall
Tom:
…And an A star.
Dan:
I mean, they gave me an A star, you know, it’s a little less understood than that. I’ll find out what I got, I can’t remember right now.
Tom:
I’d would like to point out that I got an A star for geography at GCSE and I don’t think I did much work either to be honest.
Dan:
We are not into nerd bashing, we celebrate intelligence, so well done for that Tom.
Tom:
Yeah, yeah, natural talent.
Dan:
Cool, well thanks everyone for listening. Music by Encompass Sound. All links in the show notes. Say, you can also keep checking out our blog. There’s going to be lots on prototyping this month. Check out last months. Tell you how to start with an idea.
Tom:
Yeah, new ideas and how to get going with your new business venture.
Dan:
There’s the theme building as well. Next month’s could be about what to do after you’ve prototypes.
Tom:
It is going to be about that.
Dan:
It is going to be about that, well, we know that. I was just adding a bit of tension to the listener.
Tom:
Wow. The suspense.
Dan:
Yeah, that’s exactly it. Suspense rather than tension so that we know you’re not tense right now. You can just relax in this.
Tom:
Yeah, after listening to this you must be super relaxed.
Dan:
Yeah, get out of that beat bar and go and prototype.
Tom:
What’s that thing called when you get in… something tank?
Dan:
Isolation tank..
Tom:
Isolation tank, yeah.
Dan:
With our podcast.
Tom:
Best enjoyed in an isolation tank. The Lighthouse London podcast.
Dan:
Absolutely.
Tom:
So next month it will be MVPs. That kind of bag and things based around an actual product going online, finding customers, that works. That’s been researched, that’s going to be blinding.
Dan:
It is going to be great. So until then, check out the blog.
Tom:
Wearelighthouse.com/blog
Dan:
Tweet @lighthouse, get on the face books and we will see you in any of those places.
Tom:
We certainly will.
Dan:
See you then!
Tom:
See ya!