Call Centre Metrics

Here’s a nice little source for ideas on how to measure telephone service:

http://www.the-cma.org/?WCE=C=47%7CK=227275

it explains 10 call centre metrics, amongst which:

Abandon Rate: the number of calls that hang-up before connecting to an agent. This number does not include those calls that receive a busy signal.

Calculation: Abandoned Calls / Total Incoming calls

Service Level: usually defined as the percentage of calls answered within a predetermined number of seconds.

Average Speed of Answer: (commonly referred to as ASA) is the average number of seconds it takes for a call to be answered.

First Call Resolution: measures the percentage of customer issued calls that are resolved the first time.

Calculation: Number of FCR calls / Total Number of Calls

Metrics for Telephone Calls

Here's a short overview of what I could find in terms of metrics used to measure telephone calls in organizations of the federal government. I based my findings on DPRs or Annual Reports. Only a surprisingly small number of organizations actually talk about the performance of their call centres.

Canada Revenue Agency:
Based on their 2005-06 Annual Report, caller accessibility measures the percentage of callers who succeed in reaching the telephone service, which basically means that the caller gets into the queue. Telephone service level is the percentage of calls answered within 2 minutes of entering the queue.

Service Canada:
Based on Service Canada Annual Report 2005-2006, Appendix 4 (Performance Scorecard) on page 37, it measures 3 things: the percentage of calls answered within 180 seconds (btw that’s 3 minutes guys… plain language writing…), percentage of Automated Telephone Information Service Usage (no reference to a page that explains how that works), % of calls blocked (EI High Volume Message) (which I take it means a busy signal).

Canada Business:
Based on their 2004-05 Annual Report, they seem to measure the volume of calls and do make a difference between automated and officer-assisted calls. They do have service standards, but they are very lax in terms of metrics, for example, their telephone service standard is: “Telephone service is available free of charge, generally from 9:00 a.m. to 5:00 p.m., Monday to Friday (except on holidays as they apply in each province). For exact service times, contact the centre in your province or territory. TTY is available for the hearing-impaired.”


Here are the links to the reports I used:

CRA Annual Report to Parliament 2005-06: http://www.cra-arc.gc.ca/agency/annual/2005-2006/performance-e/ar_2005-06_htmlTOC-e.html

Service Canada Annual Report 2005-2006:
http://www.servicecanada.gc.ca/en/about/reports/ar_0506/pdf/ar_0506.pdf

Canada Business Annual Report 2004-05:
http://www.canadabusiness.ca/gol/cbec/site.nsf/vDownload/annual_reports/$file/canada_business_2004-2005_annual_report_en.pdf


Now for added fun, take a look at how dropped calls on automated lines are integrated in the measures. It would seem that most of the time, a call dropped on an automated line is considered as answered, the assumption being that the caller got the information he was looking for while going through the menus, or listening to the messages played while on hold.

Measuring the Effectiveness of a Government Website / E-Service

Nielsen/NetRatings is now according more value to the time spent on a website than to the number of page views for it’s ranking of website popularity.

http://www.nielsen-netratings.com/pr/pr_070710.pdf
http://www.beet.tv/2007/07/nielsen-exec-de.html
http://www.marketingpilgrim.com/2007/07/the-problem-with-measuring-time-spent-on-a-web-site.html
http://www.iht.com/articles/ap/2007/07/09/business/NA-TEC-US-Online-Measurements.php

OK, from a governmental point-of-view we don’t really care about rankings. But that doesn’t mean we can’t learn from what the private sector is doing. What Nielsen/NetRatings is trying to do here is improve the metrics they use to measure the effectiveness of a website in attracting visitors. That goal, attracting visitors, makes sense in the context of the private sector. The goal of a government site is different. The goal of a government website is to offer information and services. In a government context you just want people to find what they are looking for as fast as possible. Actually, that’s closer to what they want. They are using your site, they are not enjoying it. You don’t want them spending more time than they need to on your site or, chances are, they’ll get frustrated and leave (and I won’t get into the impact that has on perception and image). They might choose to use another channel, telephone or in-person service, but they’ll most likely be frustrated, rightfully so because they’ll have wasted time looking on your website where the information -should- be.

OK… metrics. How do you measure the effectiveness of a website / e-service in a government context?

Clearly time doesn’t work: if they stay a long time on your site, your site is too hard to navigate, too wordy or too complicated (think plain language writing), if they don’t stay long at all, they probably haven’t found what they were looking for, or got frustrated, because your site is too hard to navigate, too wordy or too complicated.

Page views: have you ever visited a government site? You don’t get anywhere with 2 clicks of the mouse on a government site. You have a title page, and a main/portal page, and sub-pages, and how many menus? Anyhow, obviously, the most visited pages will be the top pages. But there is no information on those pages.

Users: it’s a nice statistic to have, the number of people who visit your site. But that doesn’t mean much in the government context. It’s nice to track its change from year-to-year. You can make pretty graphs with that. But the relative number of visitors depends on your mandate. I doubt the smaller or more specialized entities get much traffic. It doesn’t mean they are not effective, it doesn’t mean their website isn’t either. It’s just that the nature of their mandate makes it so they probably won’t have many visitors, but those that need to find their site just might. And that’s what matters, citizens finding what they are looking for.

The best I can come up with is surveys, either on the pages themselves or just your plain old surveys. You just have to make sure that the people you are surveying have used your site, in its current configuration. I don’t like it, I really don’t, but that’s the best I can come up with to measure the effectiveness of a government website / e-service.

Pitfalls of Internet Access Statistics

It’s hard to find good statistics on the percentage of Canadians with access to the Internet. First, Statistics Canada changed the way they measure this metric in the last few years, making prior year data incomparable (It used to be by household if memory serves). Second: define “Internet access”; do web-enabled cellphones count? Do Blackberries? What about access in public places like Internet cafés, public libraries and schools?

Truth is, measuring how many Canadians have access to the Internet is more complicated than it used to be. It gets worse if you are trying to use that statistic to support a web-service proposal. For example, you probably won’t use your cellphone to fill-out an online form (i.e. an application for a government service or program). But you might use it to check the weather (Environment Canada). This doesn’t mean that cellphones have no value as a channel; it just means they serve a different purpose. A good example would be the ice status updates for the Rideau Canal that the NCC sends out: you receive these on your cellphone. The Internet used to be considered a single channel because most people would use it from a PC at home, but the modern truth is, it’s not, it’s many channels with different uses and different potentials for service delivery.

Another aspect to take into consideration is from where a user is accessing the Internet. Organisations have policies on what type of sites you can visit and when or if you can use the Internet connection provided to you by your employer for personal purposes. There are also the issues of security and privacy: I wouldn’t do online banking from an Internet Café and I doubt many people are installing quicktax in their cubicle and e-filing from work.

So how do you gauge the number or proportion of potential users for an online service, or the potential or predicted growth of such services?

Hard to do. The percentage of the population with access to the Internet is still rising, but likely the speed (% change from year to year) of this growth is slowing. I’d estimate that most gains now occur in rural areas as coverage expands (as service becomes available) and in the elder segments of the population, possibly also in the segments with lower incomes or education, as the prices of computers go down and the Internet becomes more of a commodity. Now this is all nice and interesting, but bottom line is, most people who wanted to get on the Internet probably have done so by now if they could.

As for potential market for a new service, you have to define how your online service will be accessed. It might be erroneous to include those who only have access to the Internet at work or via cellphone if the service you are considering is more along the lines of an application than a simple webpage with information on it. The opposite side of the coin is also true. You don’t want to double count (or more) the number of users who might have access to your service: i.e. those who possess a web-enabled cellphone and Internet access at home and at the office, etc.

Finally, just because some percentage of the population has the right type of access to the Internet (let’s say at home), it doesn’t automatically mean that they are potential users of your service: age and eligibility are two criteria I can think of off the bat that would rule out a number of potential users.