Thanks @shantnu, I should maybe have mentioned in my original post that I know this is the approach pushed by Amy Hoy and admitted my issues with it. I think that there is a place for it as a form of research to compliment others such as customer interviews. I was interested to know if others had had success using that approach or if they were just promoting something Amy had said.
My problem with Amy’s post there is that it’s mainly one big straw man argument against customer interviews. I could easily construct a similar straw man argument against doing research her way but it wouldn’t add anything to our understanding of what we’re trying to achieve. To highlight some specifics from your quoted text to show what I mean about it being a straw man argument:
Before you launch, you’ll interview people who may be potential
customers, but have not yet given you money, and are ergo not customers.
They’ve got no skin in the game
This can be true, but is not always the case. Some people actually ask people for payment during the interview process as a way to get some “skin in the game”. Other methods get them to invest something other than cash, such as their time (talking to you, possibly multiple times) or social capital (by introducing you to others). Not perfect, but sadly nothing is in defining a good product to build 
You rely on your interviewees being experts at research & development
— you trust them to identify their own pains with unflinching honesty
and accuracy. To remember, in essence, exactly what they do, all day,
every day
Not true, unless you effectively give them a piece of paper and ask them to describe what they need building or the problems they have. You are meant to be the expert in R&D, part of the skill of a product manager or entrepreneur is to ask the right questions in the right way to get the information you need. Ie “When was the last time this was a problem for you? And when before that?” etc. To clarify that this is indeed a top problem, not just something they experienced recently. Likewise “What does it cost you to solve each month?” Or “How long do you spend on each month?” and then “So how much is it worth to you to have that resolved?” to clarify if this is indeed something they need solving or not really a big deal to them. Again, not perfect and there will be times this fails but again nothing is guaranteed when researching products.
And finally
They must not care about your feelings at all — to
not be, in fact, the kindly people they must be to accept your interview
request — because they will be too friendly, too supportive, too
optimistic, too nice… which of course results in worthless data.
This is the central theme of the Mom test book (I’m not getting any money for pushing it I promise ). That you need to ask questions in a way which doesn’t allow people to lie to you out of being nice. To ask questions in a way that even your Mom would have to be honest to you.
If Amy was trying to say these things are hard so lets look at some alternatives to augment or even replace them that’d be fine. My issue is the straw man approach which tries to paint customer interviews as totally broken and pointless. Customer interviews are a useful way of finding out what your product should be, when done right. Done wrong, in the way Amy outlines they are a terrible waste of time.
Sorry for the rant! I’m happy to be corrected on anything I got wrong above
