An Inside Look at a Successful Early Access Survey

Early Access Liftoff 🚀 surveys are your chance to get rich data to inform how you build new products and features. Early access is designed to help you identify the most painful problems people in your target market have, so you can build something that customers love.

Previously, we’ve shared example questions to ask customers and potential users in an early access survey. I ran a survey using this same methodology and received over 600 responses!

I’m going to break the survey down — question by question — so you can apply the same core lessons to your own business.

Writing these surveys isn’t easy – it took five hours to create the 6-question survey! Here are just a few of the things that were debated as it was written:

• How much context do we give about what we’re thinking of building?
• Do we create a landing page? Or go right to the survey?
• Do we ask different questions depending on how respondents answered earlier questions?

There are endless questions that can be asked in any survey. The goal is to ask only the important ones.

Here’s exactly how we got to our 6-question early access survey:

The Welcome Message

You typically don’t have your product or new feature built yet when starting early access. You probably don’t even have a landing page. Your welcome message is all you have to pull people in and give them context. A short to the point message is enough to explain what you’re asking about. Giving people a little bit of information goes a long way in getting them excited and primed for your questions.

In our welcome message, we wanted to give people taking the survey enough detail (but not too much) so they understood the general topic. We also had to make sure responses were unbiased so that we could really learn about the most painful problems. The worst thing we could have done was to dive into details too soon and potentially miss key learnings.

Since this early access survey was special for Product Habits email subscribers, we highlighted the fact that people who took the survey would be learning as we built. And, that by participating in calls and discussions with us they would get a first-hand view into how we ran customer interviews.

Quick tip: “Share my opinion!”

It’s common to ask for “help” when asking people to fill out surveys. We like to ask people for their opinion. People are more willing to share their opinions so it increase your response rate. It also improves the quality of the responses.

Contact
In the survey, we didn’t ask for an email address. We already had them since we sent survey links by email to the Product Habits email list. So we used our email service provider’s mail merge functionality and made sure emails were passed into our survey tool automatically. We made the survey shorter and easier by not asking for email addresses.

If you don’t have a way to capture emails, make sure to ask for an email in your survey!

We wanted to know where respondents worked to help us choose who to interview and have that context when we contacted them. Having an email isn’t always enough – sometimes people use their personal email addresses, making it impossible to know where they work without asking.

Company and Role Information
We asked two questions to get more information about survey respondents.

Based on answers to these two questions, we could easily find trends in the responses. For example, we might learn that most Product people at companies that are under 200 people have very similar issues when it comes to document sharing and creation.

We also knew that the way people collaborate on documents and share them differs by the document purpose. People in Sales are dealing with totally different issues than people in Marketing, for example, and we wanted to be able to learn why.

Segmentation is key. It’s one of the most important methods you can use in an early access survey to get deep insights just from the survey responses. You’re looking to maximize learnings about people and their behavior. By asking a couple of key multiple choice segmentation questions you can find critical patterns that’ll help you decide what to focus on next. After analyzing the patterns between different segments, you might decide to focus on a certain type of person or size of company with your product or feature MVP.

Competitors

When it comes to tools in the document space, there are tons already.

We didn’t want to create another document creation tool. But we did want to know about the tools people were using. Knowing that would help us as we built out features like what types of documents our software would support, and what type of integrations we wanted to build.

Most people only ask questions about their direct competitors. You’ll learn things you may have missed by asking about tools that are indirectly competitive to what you’re building. This additional information provides you with a lay of the land and a core understanding of what’s popular with people who are potential future users of your product.

Challenges
We kept our question related to challenges/problems broad:

This was the most important question in the survey. It’s the only open-ended one and we knew we’d get the majority of the insights we wanted here. The answers themselves help us understand what challenges people currently experience. They also helped us choose who we wanted to conduct interviews with during the next step of the Early Access Liftoff 🚀.

There are different ways to ask this question. I’ve seen it asked in a more open way, such as “What are your biggest problems with sharing documents?” and that’s not as effective as asking about the “single biggest challenge” people have. Your goal with this question shouldn’t be to learn about all the problems each survey respondent has. You want to understand what people consider to be their single biggest challenge, the most painful thing that they are dealing with on a day-to-day basis.

By using these exact words, you get to learn about the one thing that bothers people the most. You aren’t trying to get as much as you can from each person, you’re looking for what I call the “pattern of pain.” You’re hunting for the patterns in the responses you get.

We also kept our question broad. We didn’t go in depth about the features we were planning to build to keep the survey unbiased. We had hypotheses about what the biggest challenges were, and we also had ideas about solutions. But we didn’t want to dive right into them without hearing from respondents first, in their own words.

You should never assume you know more than you really do at the earliest stages of product development.

Customer Calls

We spent the most amount of time crafting this question.

Why?

Because the single most valuable thing you can do in product development is talk to customers and potential customers to learn in detail about their challenges and processes.

By getting an idea of what each person was willing to do to help us learn and create the software, we were able to front-load the list of people we could reach out to. We could have just asked for people who were willing to be interviewed on a call. Instead, we knew there would be more phases of research. We wanted to be ready with more people to contact for those future phases as well.

There is no better way to do that than ask upfront. This is a key part of the Early Access Liftoff 🚀. Use the momentum from each phase of the process to catapult yourself to the next phase.

Based on the questions we asked, we had a list of people who were willing to participate during each stage of early access, and some even selected 4 or 5 of the ways! This method saves a ton of outreach and hustling. We were setup to complete the rest of our product research in half the time because of all the people who were ready to give us their opinion (and feedback) in 5 different ways.

Here’s why we chose each of the options we did:

Being interviewed by phone or skype (no more than 20 minutes)
Being interviewed by IM chat (no more than 20 minutes)

These two questions helped us set the expectation that we weren’t looking to waste people’s time with endless questions. We didn’t need an hour of time or even 30 minutes, we just needed a quick 20 minutes.

We know that not everyone can talk on the phone, so we included IM chat to capture more participants.

Giving feedback on screenshots or demo previews

We knew that as we built our product, we’d want to get feedback on screenshots. This is key to capture future participants.

Participating in online surveys or user testing

Not everyone is willing to do an interview, but some people are happy to take more surveys, so we included this question.

Meeting in-person (San Francisco / Bay Area) at your office

We really wanted to meet with people in person. This would help us dig even deeper into the problem and understand how people were dealing with document challenges. In-person interviews can be more dynamic since you can pick up on facial expressions and body language. Plus, we love making new friends in the Bay area. So we included this option for people.

Thank You Message

People took time to fill out your survey – so don’t forget to thank them! In our thank you, we made sure to tell survey takers that we’d be emailing them soon with more information. A message like this helps remind people that they’ve signed up for early access, so they won’t be surprised when they hear from you again about the product.

Asking the right questions in early access is so important. It’s the foundation on which you build products that people love – and keep coming back to.

Don’t forget, this same process works just as well for new features.

Take the time to get your Early Access Liftoff 🚀 survey right. It will pay back tenfold and you’ll get more and better responses to help you build the best product possible.