Fumbling with Surveys

Like everything else, opinion surveys have moved from the paper and pencil era to the internet. Many online survey tools are available to help design and distribute electronic questionnaires and compile responses. As a retired researcher, I’m skeptical when I hear online survey sites claim that ‘anyone can do it’. It would be more accurate to say, “Anyone can do it …. poorly”. I had that impression with InnPower’s customer satisfaction survey, which closed yesterday, October 31.

I accessed the questionnaire online several weeks ago and, as I proceeded through the survey, noticed that some questions were, in my opinion, flawed – one in particular stood out. The normal practice would be to pre-test the questions internally with a small group of people to check response options and if their meaning is properly understood. You’d be amazed how many different ways people can interpret the same sentence.

My disturbing feeling was that neither the survey writer, nor the client, had much training or experience in questionnaire design. That was the first flag that prompted me to look for information about who was providing this service. A flawed question will provide flawed data that may be inaccurate, or difficult to interpret. 

The survey supplier is identified on the survey web page. A review of the company’s website showed that it apparently provides public relations services (press releases, content for print media, website, and audio). There was no indication of experience with opinion surveys other than to say, “other communications services as required”. That was another flag. The website also lacked any information about specific staff or principals. This is unusual for research providers because background, training, and experience are typically key. That was the third flag. I continued to search for more information, eventually discovering that the principal of the company is a “former on-air radio personality” and the current Mayor of Bracebridge. The contact number appearing at the end of the InnPower online survey seems to be the Mayor’s mobile phone.

I have no problem with an elected municipal representative operating a separate business. I think it’s pretty common. I have trouble with the notion that the awarding of this contract for an opinion survey may not have been as transparent and rigorous as possible. Some people could speculate that the mayor’s connections in the Association of Municipalities of Ontario (AMO), and its Small Urban Caucus might have been exploited for personal business purposes. Still, if there is any fault, I think InnPower fumbled in reviewing the survey design, conducting a supplier search, and in vetting qualifications.

As far as the survey goes, I have no issues with InnPower’s level of customer service. I also give InnPower credit for taking the initiative to conduct some market research. They are ahead of those executives, (I met a few) who dismiss research as a ‘waste of time’ or ‘too expensive’. This was not the best way to conduct an independent, arms-length assessment of corporate service delivery. Personally, I have reservations about the value of some of the data collected. But three lucky InnPower customers will still win a $100 credit on their account!