In the past week, I received two invitations to participate in online surveys. One was from Capital One, regarding credit card account benefits. The other was from Trip Advisor regarding opinions of travel. Both surveys had catastrophic flaws, but both companies made it impossible to notify them of such. Why? Both sent the online survey from either a no-reply email address.
What is the reason companies send email blasts from non-working email accounts? The first idea that comes to mind is to avoid receiving bounced email responses. But nearly all modern email blast services and software detect bounces and automatically remove the recipient. These messages would never reach the original sender anyway. Is there another reason? Perhaps these companies don’t want to hear from customers who wish to unsubscribe but can’t bother to click the “unsubscribe button”. I can understand that, but are there really that many? My experience shows that undesirable emails result in either the user just clicking “delete” or clicking unsubscribe. On top of that, doesn’t your email blast software have filtering to place these responses in a non-priority folder?
There really is no good reason to not monitor an email blast return address. In fact, the downside to not monitoring it can be quite significant. In both instances here, the two companies I mentioned missed out on quickly learning about flaws in their surveys.
Capital One used a firm called Directions Research to conduct the online survey. The survey loaded fine and the speed of page loading was never an issue. However, the survey was deeply flawed for several reasons:
- It allowed no way to go back if you mistakenly clicked the wrong response to a question. This produces flawed data from the erred question forward, since subsequent questions are based on your [false] response and you are forced to answer them. This essentially can make a large part of the survey data completely false. There is simply no excuse for this. It’s an online survey, not a test, and going back to correct a mistake is a fundamental of any online survey.
- It made incorrect assumptions based on a specific response. For example, it asked about the various credit cards I owned and then assumed that my Capital One card was my primary card when I specifically answered that my Chase card was my primary card. So all of these questions that assumed Capital One was my #1 should just be thrown out, since the answers are not applicable.
- It lacked an “N/A” option as a response for any question. When a question does not apply to someone, the first thing they look for is the “N/A” option. But this survey instead wasted respondent’s time by making them choose instead from false answers like “I have no interest in [whatever]” instead of “N/A”, which simply means I have not used [whatever] but does not mean I am uninterested. The “N/A” option also provides a fallback for when you click the wrong response accidentally and then are forced to answer follow-up questions that do not apply to you. Being able to chose the “N/A” option would keep you honest and provide more accurate data to the surveyor.
- They paid a $20 Amazon Gift Card for completing the online survey. I give Capital One credit for dangling a carrot. The problem is when you pay people to answer surveys, sometimes you get uninterested folks who just answer questions randomly in order to get the prize at the end. This leads to a higher response rate, but data that is more flawed. With any survey you absolutely need to ensure high quality responses and making it voluntary with no reward is one of the few ways to easily accomplish this.
Being a nice guy and wanting the same if I was the sender, I fired off an email to Capital One at the legitimate-sounding return address the online survey came from: dr-survey@capitalone.com. I explained how their survey was flawed and suggestions for fixing it. But not only was their survey email address not monitored, it didn’t even exist:
With Trip Advisor, the survey situation was much worse – the firm they used, Cvent, couldn’t handle the load and the online survey wouldn’t budge past the first page. Even an hour after the email went out the site was struggling to load the survey, taking around 30 seconds between pages. I again decided to be a helping hand and let them know about this issue. But their return address was not monitored either:
Trip Advisor did provide a link for contacting them at the bottom of the original survey email, but this simply leads to a self-service search-for-help page with no way to contact them about their survey. It’s as if they assume their online survey is perfect and that there is no possibility you might ever need to report anything wrong with it. I am a big advocate of gathering customer feedback and this really comes across as a black eye on them.
An Online Survey Done Right
As a small business owner, don’t make the mistakes they did. If you contract an online survey out to a 3rd party, take the survey yourself. Try different responses. Make sure everything works as it should. Look for typos or misleading questions. Make sure respondents can correct an error by going back a page. Ask your survey provider how they handle peak loads. While your business is unlikely to have the email list size of a Trip Advisor, you could still be the victim of running your survey alongside a large company that takes the provider down, as what happened with Trip Advisor, leaving you with no responses but a good helping of embarrassment.
There are many 3rd party companies that can conduct surveys and customer research on your behalf. But if you wish to keep it in-house, want to save some money, and your site runs WordPress, get in touch with us. One of our specialties is creating high-quality, visually appealing surveys for customer research. Whether it’s an on-going survey (similar to a restaurant comment card) or a time-sensitive survey for an upcoming product launch, we not only handle the technology behind it but also carefully craft questions that provide amazingly useful insight to help you grow your business…and we’ll make sure you receive all legitimate replies.