The people have spoken
When the lights go out, utility customers want answers and they want them quickly. If they have to sit in the dark they don’t want to be kept in it. Most are calmed knowing that the power company is aware of the outage and has crews working on the situation.
For many utilities, the most efficient way to pass that information on to customers is via an interactive voice response (IVR) telephone system. By phoning the utility, in most cases using the same number they would call for billing information or to report a downed power line, customers can report power outages and check on service progress.
Trouble is, though IVR systems are now a standard part of communication with any large company, from credit card providers to mutual fund firms, consumers still have a hard time using them. Part of the problem is technophobia - people seem to panic when they have to "talk to a machine." But much of the blame lies with the companies that use IVRs. Many a phone system presents callers with a maze of choices that confound logic and seem designed to obstruct communication instead of easing it.
After Portland General Electric (PGE), an electric utility serving 668,000 customers in the Portland and Salem, Ore., areas, received a host of complaints in 1996 about its outage reporting system, the utility formed an eight-person multidisciplinary task force to improve the system.
Key components of the task force’s work were usability tests and focus groups, input from which ultimately helped make the system more user-friendly. "The objective was to improve accessibility for our customers," says Linda Evens, market research analyst, Portland General Electric. "The name of the project was Getting Through, because the major complaint customers had during outages was that they couldn’t get through to anyone, due to busy signals, long wait times and misdirected calls. One of PGE’s service goals is to be easy to do business with but unfortunately the complexity of the system contradicted that goal."
The focus groups were conducted by Mark Camack, vice president of the Energy Research and Consulting division of Market Strategies, Inc., a Southfield, Mich., research firm. Camack, who has conducted research on many IVR systems, says that PGE’s commitment to the research process was an important contributor to the success of the project. "Everyone on the team worked together, instead of working in isolation. And everyone was committed to improving the system and acting on the information we obtained from the research."
Usability testing
Prior to the focus groups, Chris Bond, PGE’s human factors analyst, conducted the usability testing. As Bond looked on, individual participants were asked to call a prototype of the PGE phone system and complete tasks ranging from reporting an outage to getting a service update on a previously reported outage.
The tests were conducted with customers in a range of demographic groups (Gen X, baby boomers and senior citizens) to get opinions from younger customers, who are generally more comfortable with and in some cases even prefer using an automated system, and older customers, most of whom prefer talking to a person. Bond also conducted usability tests with PGE employees and found that they were actually harder on the system than the actual customers were.
"We created a shadow of the existing system and used it as a prototype for iterative testing of the design changes," Bond says. "With usability testing you’re more closely simulating the real world, because you’re looking at the individual experience, capturing each participant’s actions and reactions. I usually run people through a series of tasks and I measure how long it takes them, the number of errors they commit, and any problems or expressions of self-blame or dissatisfaction. If they like something I note that as well."
After the testing, participants filled out questionnaires about the tasks they performed, rating various characteristics of the design and indicating what they liked most and least about the system.
Similar exercises
Customers did similar role-play exercises in the focus groups. Afterwards they discussed their impressions of interacting with the system. As with the usability tests, the focus group participants represented a range of ages and incomes. "By Chris doing the one-on-ones in advance, we were able to tweak and isolate issues before the focus groups, so that the groups were used to fine-tune what Chris had isolated in the one-on-ones," Camack says.
Focus group respondents were given index cards briefly describing the purpose of their call along with the information they needed to complete it. The descriptions were kept free of utility industry jargon to prevent respondents from listening solely for keywords during their calls.
Some were given dead-end tasks to check their reactions to how the system handled them when they didn’t supply the "correct" information. "We told them that they might be getting a twist," Camack says. "That way they might be more open to saying that something went wrong and not be afraid to say that they made a mistake or didn’t understand something."
Respondents had no time limits; they were simply instructed to come back when they thought they were done. They were asked immediately to write down their impressions before any group discussion.
During the group discussion, some of the words that the respondents might have heard while using the phone system were displayed on an easel. The discussion was designed to uncover confusion about the sequencing of menu items, the meaning of terms and to determine if certain terms went unnoticed by some respondents.
Set of guidelines
One of the most valuable byproducts of PGE’s systematic approach to improving its voice response system was the development of a set of guidelines that will be used to steer future changes to the system. "We have rules about wording, navigation and control, the dialog structure, how to provide feedback and error handling," Bond says. "For example, you don’t use a word like ‘invalid.’ It’s very accusatory and it assigns blame to the caller when maybe it wasn’t their fault. They may have just been responding to prompts from the system. We documented all of the changes we made so that if anyone asks why a change was made, I can show them why and show that it’s based on empirical research."
"They developed a system philosophy," Camack says. "In other words, what kind of experience do we want our customers to have? They felt that the customers should go no more than three or four menus into the system before they get the information they want. They also felt that no more than three or four items on any one menu were palatable and the research verified that."
The main goal is to keep the system as consistent as possible while reacting to customer needs, so people know what to expect when they call. Prior to the research, changes had been made on an ad hoc basis in response to customer complaints but without a thorough consideration of the effect the changes would have on the system overall.
Changes suggested by the research included shortening greetings and streamlining menus. "We eliminated the extra words, the padding, the conversational pleasantries, the anthropomorphizing. It’s not a person, it’s a computer and people are interested in efficiency and we try to get them through as quickly as possible. If they need to speak to someone they can do it that much quicker," Bond says.
"Chris quantified the number of words saved," Camack says. "We went from a 129-word main menu to a 45-word one, and yet, in his analysis, satisfaction rates were up, the average timing during the role play from being transferred was reduced something like 29 seconds per transaction. Multiply that by 650,000 calls per year and the manpower savings to PGE was enormous. And obviously the research suggests that the customers will be more satisfied as well."
The guidelines also helped prioritize where the various functions appeared in the phone menus. With customers making 300,000 calls each year to stop, start or move service, and 250,000 calls with billing and payment questions, it was clear which options should be highest on the menu of options.
"Good interface design is based on a body of knowledge and experience and your results from this kind of research and the testing you do, whether it’s focus groups or usability testing. It’s in documenting the lessons learned. That should be the key driver to your design decisions," Bond says.
Of course, not every change can be accommodated. That’s where having a strict set of guidelines comes in handy. "I don’t ask participants in the usability tests to evaluate the system by telling me how they’d redesign it," Bond says. "End-users aren’t designers. They’re good at helping reveal the design flaws by demonstrating the shortcomings. It’s through observing them and their reactions that I pinpointed the problems."
Ripple effect
The improvements to the system caused a ripple effect within the phone center, necessitating changes in staffing. "We had to avoid the hurry-up-and-wait syndrome," Evens says. "We had made it easier for the customer to get through the system only to be put on hold because of limited staffing. That was revealed very quickly."
Also, those who staff the phone center had to be prepared to handle almost any kind of call. "When customers call in, they expect that whoever answers the call can help. That affected our call center configuration because we now have to take a customer-based view instead of an internally focused, skill-based view. Before, it was set up so that certain teams worked on certain issues. Customers aren’t geared that way. They want to be able to get answers from whoever takes their call."
Expectations rising
As consumers become more comfortable with IVR systems, they expect them to be more and more efficient and user-friendly, Camack says. "Seniors now interact with them for their pensions, their banking transactions, etc., so there is a lot less blatant fear toward the systems than there was five years ago. But their expectations are rising because they see them as two-way communication vehicles. In the early days of energy utilities, customers felt happy just to get through and report that their house is out of power. Now, people expect to be able to find out more and more information."
But Camack warns against crowding the system with a ton of nice-to-have options. "You have to walk before you run. Put things on the system that you know work well and the customers like and as they become more accustomed to it then you can add additional functions. You have to be careful because when customers have a bad experience with one part of the system they feel the whole system is bad."
At its core, a utility’s phone system must give callers solid information. "They want to know what the power company knows and what it doesn’t know. They’d like a time-stamped message so they know how current the information is. If the utility is going to include estimates of repair times, respondents have said the utility should overestimate the amount of time because they’d rather have their expectations exceeded than be disappointed," Camack says.
He also recommends that the utility have someone on staff whose job it is to keep track of customer perceptions of the voice response system. "Especially in this day and age, when electric utilities and energy utilities are so brand-conscious as they get ready for deregulation, your phone system sends hundreds of thousands of brand impressions about your company each year. The choice of the voice talent you use, and the manner and tone and words used can imply so much about your company. It’s important for someone there to keep track of how customers are interacting with the system."