Build or validate?

Discussions
#14

Thanks for your feedback Reshal.

I remember feeling really excited when I read that post. I even tried to put that strategy in practice (only with signups, never added the “Pay” button).

I didn’t work for me. Getting some traffic to a landing page never felt reliable enough to me.

What if people that came there are not from my target market?
What if the number of people is too small to consider conversion rates reliable?
What if the design or copy of the page is the reason that makes people not convert?
What if the solution offered requires the users to test it before they can decide if they want to purchase or not?

Some months later, I found this great post by Dan Norris. It specially resonated with my experience:
http://wpcurve.com/is-startup-validation-bullshit/

1 Like
#15

Yes, we’re talking about the same course.

The idea of the creative atom is to get something out fast and get feedback on that, but you still need some time to build a product during which you run the risk of being isolated from feedback.

What I’m saying is it can’t hurt to be proactively reaching out to your target audience to get more feedback (hence the customer development interviews that Amy doesn’t like). People reacting to what you’re building (be it blog posts, ebooks, software) can help but often times it’s not enough to get enough feedback, so you should reach out rather than avoiding people.

#16

Thanks for your insights @ShawnArnwine.

My actual process of finding product ideas is:

  1. Defining some requirements I want my product to follow (SaaS business model, minimal interface, extremely easy installation/integration, revenue from day 1, that can be built and sustained by a one or two people team…).
  2. Finding a clear target market that is already paying for similar products (Shopify store owners…).
  3. Finding an existing task that my target market is dealing with (customer support).
  4. Thinking about a simple way to improve that task (offering easier access to relevant data about the customer).
  5. Thinking about a really simple solution that fits the previous criteria and that I feel excited to build and proud to market (v1 of my possible solution).
  6. Research the market to see if there’s people trying to solve the same problem, and if I still think my product can offer something different/better.

Step number 6 is the one paralyzing me now.

My gut is telling me that I should build a minimum but polished version 1 and start the iteration process based on the reactions of customers respect the product.

My head is telling me that I should get some feedback from potential customers first, but I don’t find a strategy that I feel will give me reliable enough data.

#17

Good point @tommy_jarnac.

I’ve been tempted to join 30x500 a couple of times. Amy’s approach makes a lot of sense, at least in theory. I guess it makes my rational mind happy to think that I can remove so much risk only by observing methodically.

My problem with Amy’s approach is that I’m only interested in building a product if it fits certain requirements. That can be incompatible with starting with a problem and having to completely adapt yourself to solve that problem.

I like how you describe a possible balance. We’re often too worried about having clearly defined strategies we can follow step by step, but maybe it’s ok to organically combine observational market research, customer interviews and simple experiments.

#18

How have you been trying to get traffic then? Because it’s exactly important to get the traffic from your target market, and Twitter has been a really good companion at achieving just that.

#19

There are obviously a few ways to slice this and different approaches that will work…and I agree that there isn’t a hard and fast way to figure this out. Just to further clarify what has currently worked for me, albeit a small win thus far, has been a combination of the finding a pain and talking to people to further refine my solution and determine if they will pay for it.

Instead of coming from a solution first approach (which always tripped me up), I decided to find the pain and determine if it was worth solving. So I went out to applications that I currently use and scoured their forums looking for a consistent pain point.

A great place to start is in the “feature requests” category, most larger applications have this.

While checking through the Box.com forums I found a request that had been out there for years and that Box doesn’t seem to have on their roadmap. So I marked this down as a possibility. About a week later I happened to have lunch with a friend who works for Box and we were talking about ideas…and he said that he knows of Box customers have the exact problem I found…he had no idea about the forum request. That’s when I knew I had to look into this further.

So I set up a landing page to capture emails and posted the link in the forums. I got a total of 50 sign ups in a matter of days. Not a ton by any stretch, but enough to see there was interest. I then emailed each one individually and had conversations with them…mostly over email (which isn’t ideal) but then a few over the phone.

One other thing to keep in mind, if you do explore platforms, make sure people are paying something to use it or that you can get to their paying customers. Box for example gives so much away for free that the challenge for me has been identifying paying customers, but I’ve slowly managed to do it.

2 Likes
#20

“If you don’t do your job getting real data, you might as well get data that has a high likelihood of being false to fill out your spreadsheet.”

Yo, everybody. Hi hi hi hi.

The 30x500 process is not as Tommy is describing at all.

The 30x500 process is exactly the opposite of working in a vacuum: You study the audience. You interact with the audience by helping them in a natural setting (e.g. a forum, like I’m doing right now). You teach the audience. You get them to sign up for your mailing list. You find & foster natural sources of data — when the audience is doing its thing — you don’t give them an incentive to behave differently, and you don’t rely on faulty self-reporting. (There’s a reason social science research studies rarely rely on self-reporting. It’s because it’s very low accuracy.)

If you ask people “What’s your biggest problem?” or “What’s the secret to your success?” you will almost invariably get different answers than when you observe.

For example: Tommy says his problem is 30x500 taught him to work in a vacuum.

Meanwhile, observation says: Tommy, you took 30x500 >2 years ago and didn’t reply to almost any of the “do this and share your results” homework threads. Of 12 weeks of class, I only see your work for exercises 2, 4, 5 and 7.

No denying that 30x500 has gotten much much better since then — and, as an alumnus, you have access to all the new lessons for free — but even in 2012, the process still focused on starting with real data (Safari) and ebombing, rinse and repeat.

If you feel like you’re in a vacuum, I’d advise you to go watch and try the new 30x500 Bootcamp video lessons that show clearly the entire process and demonstrate the specific Safari & ebomb techniques (again). You may find that after all this time, you’ve misremembered.

3 Likes
#21

Hey Amy, thanks for chiming in.

I AM NOT saying “30x500 taught me to work in a vacuum”. I was just saying that you can still have gaps in your understanding of what your customer needs, even by following 30x500. Interacting with them by the way of blogs and forums still gives you an incomplete picture. It is not like having them one-to-one and getting to understand their life as a person.

I agree with you in the way you should not ask stupid questions such as “what’s your biggest problem?” or “what’s the secret to your success?”. That’s not what customer development is about. You have to understand what job your audience is trying to get done, and how a product could help them get this job better and faster.

If you have found out a pain around sending invoices for freelancers by doing the 30x500 sales safari, you can go interview a couple of freelancers and learn how they actually do invoicing. Ask them to tell you stories, not opinions. ( example: “How did it go last time you sent an invoice and it didn’t go as you wanted?”). People are good as telling stories but not necessarily in understanding the root of their problems without being prompted.

The thing is most of your audience actually shies away from talking to people (hey, we’re developers after all) and so they feel more comfortable staying behind their screen and “doing research”. I think this is a bad habit to keep and actually breaking out of your comfort zone and talk to some people will teach you lots of things about these people and bring more insights into how to develop a product.

But that’s my opinion, and you’re right in that I haven’t checked 30x500 in a while and will have a look at what it becomes now.

1 Like
#22

That is one of the mistakes I made. I didn’t have a clearly defined target market, so I used my social network to test the idea. Fail.

I like Twitter approach. I was thinking about using http://lukewarmemailer.com. Have you used it?

It even offers you the option to obtain email addresses from Twitter profiles you set as relevant to your target audience.

1 Like
#23

Wow. Haha. Never seen a tool like that, directly integrating into docs. Looks cool, and like a good business!
But you should exactly use Twitter to find people within your segment and demographic. Maybe just go ahead and strike a conversation with them over Twitter, a friend of mine has had good success with that. Lot’s of Twitter tools are free up to a certain amount / or for a few weeks, you should be able to get some idea of it will work for you before eventually paying for the product.

#24

Find Twitter followers emails using the Interwebz and an army of cheap workers, and then spam them about your GREAT AND WONDERFUL Product, that will TOTALLY CHANGE their life forever.

Yeah, don’t see how that can go wrong. :smile:

2 Likes
#25

How do you promote your book + courses @shantnu? Thanks!

#26

@CescVilanova I’m the wrong person to ask, as I’m mostly an amateur myself. I haven’t done much marketing as I’ve been spending every free minute trying to finish the two books I’m writing. I do plan to restart marketing once I’m finished.

Mostly, I’ve been using blogs. Like all marketing, it doesn’t always directly translate to sales.

But it is a great approach for long term planning. I’ve got a website and a book I was writing which I had abandoned several months ago. But I still get hundreds of visits a day from organic sources, and I’ve decided to restart that book.

This is where Amy Hoy’s approach works very well. She recommends studying people in their natural habitat. It tells you not only what problems people are facing, but how you can reach them as well.

My first courses were a bit haphazard. For my next course, I plan to be a bit more scientific. Maybe I’ll share the results here.

Edited to add: Forgot. I had done some Google, Twitter and Reddit ads as well. Google ads didn’t do too great, Twitter ads were better, but then I wrote a blog post and pointed the Twitter ads to them. At the end of the blog, I had a link to my book. Most people who read the blog ended up buying.

Reddit ads did the worse. I actually got insulted and downvoted for bringing my greedy commercialism into the high brow culture of Reddit, even though the ads were in a tab called “Promoted” that you had to specifically click to see.

1 Like
#27

Reddit seems to have turned into a bit of cesspool, with the owners prizing free speech (including hate speech and harassment) above community. I’m not sure if they have done this out of (IMHO misguided) principal or laziness.

#28

It’s actually worse. Google and Twitter show your ads to everyone (unless they have an ad blocker). On Reddit, ads are hidden on a special tab called “Promoted” that you have to specially click just to see ads, and that hardly anyone (afaik) clicks on.

When I was buying ads, I found it hard to get slots, as slots for the next two months were booked. I’m guessing they are raking so much money, they don’t give a damn either way.

Trying to validate a problem before building a solution I'm falling in love with
#29

Promoted shows me an entire page of ads, but there’s also A promoted link at the top of every page. Is that not also an ad?

#31

Great questions.

First of all, if you can do it manually and sell it as Dan Norris did then that’s your best bet for true validation, but I’d still try to pre-validate it a bit just to you understand the pain and how the customers perceive the pain better. That’s valuable info when you start marketing it.

  1. I’ll take the second one first: How would I know that their “will not buy” feedback is accurate since they can’t try it out.
    Most software companies I know say that they do much better offering a money back guarantee than a free trial. Also, some (many?) customers will buy it without trying it. Therefore, you should be able to sell some folks on it just with the marketing information before you have a product. If you can’t do that and you have a decent sample size such that this reflects what’ll happen with ramped up marketing that’s a bad sign that you won’t be able to sell to people who can’t “touch it” first. That suggests the marketing/sales copy or the product itself is not doing it’s job.
  2. How do I know their YES is accurate?
    You could interview them about the pain. Find out why they said yes.
    How do they solve the problem now? How much time $ and effort does that cost them?
    Make them sell you on the idea..
    The more you probe the more you’ll see whether it was a “flip answer” : “yeah, i’d buy that”. It also tells you more about why they’d buy it. So if they say “yeah, I’d pay”, you could ask “have you searched for a solution before?” if not, why not? (pain must not be too great). You might also find out more about potential objections: "How would I find someone who knows shopify really well ? (Aha! You may need to address this in your marketing).

You could also ask:Will you tweet this (or “would you refer a friend to this”)?
It’s an indirect way to gauge just how interested they are. (That also has the benefit of (I forge the psych principle ) that if someone PROMISES to do something they’re much more likely to do it.
This doesn’t guarantee they’ll buy it.

How about actually pre-selling it with a money back guarantee? Works for KickStarter (but you save the 30% commission)

Validation is far from perfect. It’s like vetting a stock you’re about to invest in. There are not guarantees, but it might keep you from choosing Enron over Apple. I.e., it might help you choose the better idea to move forward with.

#32

Lot of great points in this thread…

My $0.02 is nothing is truly validated until you have paying customers who are getting value from the product.

I do think holding targeted conversations with ideal customers prior to building anything is an important step. But the goal there isn’t to officially validate or invalidate an idea. Goal of these conversations is:

A) See if the basic idea even resonates at all (“I get it!”) or not (“huh?”).
B) understand why the problem matters so much in the customers eyes.
C) nail down the single most important piece that makes the product valuable.

Assuming the idea does resonate, and you understand the problem/solution well, then you’ll have confidence to move onto the next step.

Next step could be building a very minimal version of the product (just build that one thing you learned in C above), or if possible, deliver that as a manual service at first. This gets you to paying customers fairly quickly, or if it flops, then you haven’t spent too much time (but you probably learned a thing or two that could lead you elsewhere).

Bottom line: The idea that you’ll have 100% confidence the product is validated prior to building something or launching is kinda BS in my opinion. It’s all educated guesses until it’s in the hands of paying customers.

3 Likes
#33

Nice insights, thanks @CasJam.

About this:

Next step could be building a very minimal version of the product (just build that one thing you learned in C above), or if possible, deliver that as a manual service at first.

How final/professional/polished do you think this needs to look like?

I’ve sometimes felt than going “manual” can make potential customers say “no” for offers they would maybe said “yes” if the presentation and delivery of the value proposition felt “final”.

#34

It really depends on the nature of the product…

I think there’s a difference between a crappy first version of a product and a truly minimal/focused version that while it’s feature-set is limited, is still an elegant solution.

Most new products have a handful of potential features / capabilities. During your early discussions with prospects, you might discuss features X, Y, and Z, but you’ll learn that Y is that one feature that’s make or break. The rest are nice-to-haves.

So that would tell you to build a first version that only does Y. Get it in the hands of free beta users, test, get feedback, and if they are truly getting value from it, they should be ready to pay for it.

If possible, I’d look for ways to deliver Y manually, but this isn’t always an option (depends on the nature of the product). Maybe it means investing a bit to develop that focused version 1, and deliver the nice-to-have features manually.

I’ve sometimes felt than going “manual” can make potential customers say “no” for offers they would maybe said “yes” if the presentation and delivery of the value proposition felt “final”.

If you’re solving a problem (or delivering a clear ROI), then the client should be willing to pay for that (clients pay consultants to solve problems manually all the time). That said, not everyone will go for it in this early stage. So it becomes a numbers game. Talk to enough early prospects to get to a handful of true “early adopters”.

1 Like