We sometimes get questions from customers who ask why we settled on a 4 point response scale and why we don’t offer a mid-point ‘average’ option. When we explain it, most conversations end positively. We thought it would make sense to spell out our rationale publicly and explain why we don’t ‘do average’!
Customer Thermometer provides a survey solution designed to get high, and therefore more representative, response rates. It has been created to give users real time, actionable insight into their customers.
Whilst the Customer Thermometer app remains simplicity personified, at its heart is something incredibly powerful.
Can there be any more important metrics inside a business, than understanding which of its customers are happy or disappointed?
The service has stopped customers from leaving our customers and response rates are often hundreds of percent higher than traditional customer survey methods.
One of the main things powering both the response rate Customer Thermometer generates and its ability to help businesses keep their customers is the 4 point response scale.
There is much academic debate as to whether even or odd numbered scales are the most effective. Some studies find support for excluding it while others for including it depending on the subject, audience and type of question Importantly, no one scale is favoured over another, with academics stressing both scales are valid and statistically reliable, and that companies should pick what best suits the task in hand.
We are in the business of feedback, but more specifically, we’re in the business of stopping customers from leaving and helping our customers to uncover what they otherwise would never have known. A silent customer or a customer who doesn’t choose to offer an opinion is dangerous and could quickly, without warning cease to become a customer. Whether that’s someone spending $1,000 with you or $1m/month – the silence still matters.
Our software’s job is to get customers to speak up.
For all these reasons and more we built a system around a 4 point scale, with no ‘neutral point’.
A 4 point scale, cuts decision fatigue
One of the reasons we developed Customer Thermometer was to begin a crusade against the 20 question, radio button style survey.
We’re the first to agree that when conducting market research or an in depth annual survey, multi-question surveys can be a necessity. However, for sensing customer satisfaction in a busy B2B environment, they are dangerously unrepresentative and potentially annoying to customers.
Why? Because no one has time any more. As a result, industry standard response rates for that type of survey are miniscule, typically sub 2%, and when it comes to asking customers how they are feeling in order to fix service issues, you need as much feedback as you can gather.
A four point scale encourages customers to quickly feed back on how they felt about a particular service. In our testing prior to creating the service, a 4 point scale significantly outperformed a 5 point scale in response rate when it came to rating service.
Adding a middle ground / average button in the middle of our four, provides something else for the customer to consider… it makes them hesitate, become uncertain and potentially in that split second, give them too much to think about. In other types of survey, many hit the neutral button out of ease alone, leaving you with potentially misleading data. With a neutral point added, the decision on which button to click can become more of a prolonged process, as opposed to a natural reaction.
Often, when this hesitation occurs, either nothing gets clicked, or the middle ground gets clicked and the opportunity to solicit actionable feedback is lost.
Remember, Customer Thermometer is only ever sent to customers who have had an experience with our users’ businesses. It is therefore much less likely that they will have no opinion on the service undertaken.
Customer Thermometer’s satisfaction feedback approach
Clearly it is impossible to recommend a single perfect response scale for all customer survey usages, as the optimum scale must be moulded to meet the survey’s requirements. We provide flexibility for the labeling of the points on the scale in all Email Thermometers to enable users to have flexibility in their nomenclature of the 4 points.
Odd numbered scales are generally regarded as allowing for a ‘neutral’ option such as ‘neither agree or disagree,’ or ‘don’t care.’ However, much of this academic debate centres on the reliability of these surveys. i.e. would the same survey sent to a different set of the population produce the same result reliably? In the case of the B2B customer satisfaction sensing Customer Thermometer carries out, we are not seeking this kind of reproducibility, we are seeking specific sets of individual feedback so that our users can act upon them whilst providing a benchmark.
Supporters of the neutral point argue that giving a ‘don’t know’ option ensures that respondents do not manufacture opinions instantaneously.
However, advocates of even numbered scales, which include Customer Thermometer, argue that in reality people are never neutral on issues and always have an opinion, even if they had not previously conceived of it. Moser and Kalton (1972:344) argue that ‘there is clearly a risk in suggesting a non-committal answer to the respondent,’ as they believe that a mid-point allows respondents to ‘opt out’ which in turn provides uninformative data. Andrew Messing at Harvard University has stated of midpoint responses, “you get to pretend that this response (like the others) enables you to measure a response.” Many academic studies have concluded that offering a midpoint leads to clustering at that midpoint, (Krosnick, Narayan, & Smith. 1996).
The advantage of a scale, like ours which forces a view is that you are not left with a lot of ‘middle ground’ responses where you’re not sure what they meant (they don’t care, they don’t know, they have no view on it, they can’t be bothered?) and you have specific actions you can take in each case.
Can customers be 100% ambivalent about a service?
Following on from our argument above, we genuinely don’t believe a customer can be ambivalent about an experience they’ve just received. An expression of how someone feels about customer satisfaction is based around emotion – it’s a gut feeling. Generally, it’s not something that needs to be considered or thought through in any detail.
If you’ve just had your house cleaned, come off a conference call, returned from a hotel visit or had an experience with a customer service rep, you will have an opinion on that experience which will either be in some way positive or negative. It just can’t be completely average. It may not have been great (yellow)… it might have been awful (red)… it might have been ‘fine’ (green) or it might have been excellent (gold). We believe that “OK” is not at all the same as “completely neutral”.
4 points drives action and service recovery within the user’s business
The Customer Thermometer survey approach has been designed from the ground up to drive service recovery action. Most surveys are never acted upon in real time – and one of the main reasons for this is that neutral points give a business nothing to act on.
Say a customer clicked an ‘Average’ on Tuesday at 09:45… his ticket number was 1002356 and the agent dealing with the ticket was David (and yes Customer Thermometer can tell you all that).
The problem here is that an ‘average’ rating gives you no next step because you have no information about how that customer is feeling.
Not providing an average rating drives a workflow around service recovery and it helps businesses to uncover issues, bring them to the surface and make continuous improvement.
In many survey contexts – it would absolutely make sense for a recipient to be provided with a middle ground… Customer satisfaction ratings are fundamentally different to opinion-related surveys in our view.
Take for example the question “Do you feel your taxes were well spent in the last financial year?”
Many people will have ‘no opinion’ on that subject. Some will strongly agree, some not so much. With this type of opinion sensing – we absolutely understand that it’s possible to be ambivalent, to not have an opinion, to not care, or to not know enough to be able to express an opinion.
Our customers typically ask questions such as:
“How was your experience with us today?”, “How was our support team today?”
To answer this type of question – or any other type of satisfaction related question, we don’t believe it’s possible for someone to truly provide an ‘average’ or “don’t know” response because our surveys are sent to someone who definitely experienced our users’ service. The response has to have an opinion attached to it, however slight.
For all these reasons, we’ve settled on our 4 point scale and have thousands of customers using it who have embraced the service and are using it every day with great effect.
Whilst there are persuasive arguments in both directions (and many others directions such as 3, 7, 10 and 11 point scale) our experience firmly shows that in order to elicit meaningful, actionable and useful customer satisfaction feedback, there should be no midpoint.
At its heart, Customer Thermometer is all about getting a firm measure of satisfaction, whilst also providing real time, insightful results, driving action and stopping customers from leaving. We believe the 4 point scale best supports these aims.
References and sources
- Schneider et al, Stanford, 2008
- Moser and Kalton, 1972
- Krosnick, Narayan, & Smith, 1996
- Chang, 1994