Measuring the Effectiveness of a Government Website / E-Service

Nielsen/NetRatings is now according more value to the time spent on a website than to the number of page views for it’s ranking of website popularity.

http://www.nielsen-netratings.com/pr/pr_070710.pdf
http://www.beet.tv/2007/07/nielsen-exec-de.html
http://www.marketingpilgrim.com/2007/07/the-problem-with-measuring-time-spent-on-a-web-site.html
http://www.iht.com/articles/ap/2007/07/09/business/NA-TEC-US-Online-Measurements.php

OK, from a governmental point-of-view we don’t really care about rankings. But that doesn’t mean we can’t learn from what the private sector is doing. What Nielsen/NetRatings is trying to do here is improve the metrics they use to measure the effectiveness of a website in attracting visitors. That goal, attracting visitors, makes sense in the context of the private sector. The goal of a government site is different. The goal of a government website is to offer information and services. In a government context you just want people to find what they are looking for as fast as possible. Actually, that’s closer to what they want. They are using your site, they are not enjoying it. You don’t want them spending more time than they need to on your site or, chances are, they’ll get frustrated and leave (and I won’t get into the impact that has on perception and image). They might choose to use another channel, telephone or in-person service, but they’ll most likely be frustrated, rightfully so because they’ll have wasted time looking on your website where the information -should- be.

OK… metrics. How do you measure the effectiveness of a website / e-service in a government context?

Clearly time doesn’t work: if they stay a long time on your site, your site is too hard to navigate, too wordy or too complicated (think plain language writing), if they don’t stay long at all, they probably haven’t found what they were looking for, or got frustrated, because your site is too hard to navigate, too wordy or too complicated.

Page views: have you ever visited a government site? You don’t get anywhere with 2 clicks of the mouse on a government site. You have a title page, and a main/portal page, and sub-pages, and how many menus? Anyhow, obviously, the most visited pages will be the top pages. But there is no information on those pages.

Users: it’s a nice statistic to have, the number of people who visit your site. But that doesn’t mean much in the government context. It’s nice to track its change from year-to-year. You can make pretty graphs with that. But the relative number of visitors depends on your mandate. I doubt the smaller or more specialized entities get much traffic. It doesn’t mean they are not effective, it doesn’t mean their website isn’t either. It’s just that the nature of their mandate makes it so they probably won’t have many visitors, but those that need to find their site just might. And that’s what matters, citizens finding what they are looking for.

The best I can come up with is surveys, either on the pages themselves or just your plain old surveys. You just have to make sure that the people you are surveying have used your site, in its current configuration. I don’t like it, I really don’t, but that’s the best I can come up with to measure the effectiveness of a government website / e-service.

Pitfalls of Internet Access Statistics

It’s hard to find good statistics on the percentage of Canadians with access to the Internet. First, Statistics Canada changed the way they measure this metric in the last few years, making prior year data incomparable (It used to be by household if memory serves). Second: define “Internet access”; do web-enabled cellphones count? Do Blackberries? What about access in public places like Internet cafés, public libraries and schools?

Truth is, measuring how many Canadians have access to the Internet is more complicated than it used to be. It gets worse if you are trying to use that statistic to support a web-service proposal. For example, you probably won’t use your cellphone to fill-out an online form (i.e. an application for a government service or program). But you might use it to check the weather (Environment Canada). This doesn’t mean that cellphones have no value as a channel; it just means they serve a different purpose. A good example would be the ice status updates for the Rideau Canal that the NCC sends out: you receive these on your cellphone. The Internet used to be considered a single channel because most people would use it from a PC at home, but the modern truth is, it’s not, it’s many channels with different uses and different potentials for service delivery.

Another aspect to take into consideration is from where a user is accessing the Internet. Organisations have policies on what type of sites you can visit and when or if you can use the Internet connection provided to you by your employer for personal purposes. There are also the issues of security and privacy: I wouldn’t do online banking from an Internet Café and I doubt many people are installing quicktax in their cubicle and e-filing from work.

So how do you gauge the number or proportion of potential users for an online service, or the potential or predicted growth of such services?

Hard to do. The percentage of the population with access to the Internet is still rising, but likely the speed (% change from year to year) of this growth is slowing. I’d estimate that most gains now occur in rural areas as coverage expands (as service becomes available) and in the elder segments of the population, possibly also in the segments with lower incomes or education, as the prices of computers go down and the Internet becomes more of a commodity. Now this is all nice and interesting, but bottom line is, most people who wanted to get on the Internet probably have done so by now if they could.

As for potential market for a new service, you have to define how your online service will be accessed. It might be erroneous to include those who only have access to the Internet at work or via cellphone if the service you are considering is more along the lines of an application than a simple webpage with information on it. The opposite side of the coin is also true. You don’t want to double count (or more) the number of users who might have access to your service: i.e. those who possess a web-enabled cellphone and Internet access at home and at the office, etc.

Finally, just because some percentage of the population has the right type of access to the Internet (let’s say at home), it doesn’t automatically mean that they are potential users of your service: age and eligibility are two criteria I can think of off the bat that would rule out a number of potential users.

The Web/Internet as a Service Delivery Channel

With the advent of government online, the measurement of the effectiveness of the Web as a channel of service delivery becomes more important, as a greater portion of the population gains access to the Internet and as the services offered online become more substantive.

The presumed cost efficiency of the Internet as a means of providing services to citizens is one of the main factors driving the push to web services. The added flexibility (anywhere, anytime, anyplace) gained by the population in accessing services is another.