Latest Articles

Business Briefing: Where Next for Venture Capitalists?

Despite the economic downturn venture capitalists are still looking at investment opportunities in digital media. New Media Knowledge spoke to one to get the low down on what VCs want.

more

Social Media in Practice: BBC Radio 5 Live

Web publishers and broadcasters always have to look for new and innovative ways to maintain existing audiences and win new ones. When BBC Radio 5 Live realised it was losing listeners to other sources of football-based content it launched a new service to win them back. New Media Knowledge met the people responsible.

more

Conservationists Embrace Social Media

Television is increasingly embracing new media to reach new audiences and add interesting applications. New Media Knowledge talks to the people behind a new online conservation series that aims to raise wider awareness of the plight of endangered species.

more

• The Language of the Web
• Rough Guide to: In-Game Advertising
• Alexander McCall-Smith Engages Web 2.0
• Business Brief: Video Advertising Looks to Future
• ‘Virtual Home’ for Ex-Pats in London Established
• Rough Guide to: Social Media and the Law
• Record Visits for UK Blog Sites

Related Articles

• Accessibility for Better Business

  Businesses of all sizes are missing out on opportunities to create new leads and make more sales because they aren’t following basic web accessibility practices. That’s according to a new white paper from CMS developer Squiz. Ian Delaney reports. more

• Usability: Whose Problem is it Anyway?

  Trocadero's CC Club was host to a discussion about usability in the current new media landscape last Monday. Usability: whose problem is it anyway? attracted a set of lively industry practitioners seeking enlightenment on their path to creating usable and accessible websites. Cynthia O'Murchu reports. more

• Expert usability review vs. usability

  Lisa Halabi outlines the difference between the two usability evaluation methods: The expert usability review and usability testing. more

• Improve e-commerce with filter & sort

  Filtering and sorting are essential for helping users find the products they're looking for. How can you make best use of this essential functionality, asks Jonathan Webb more

• Forget guidelines talk to users

  Accessible and usable websites are a high priority within the public sector, yet within the private sector many feel overwhelmed by all the guidelines. Organisations should stop worrying about official recommendations and simply start talking to disabled users, argues Stefan Haselwimmer more

Practical Usability: Beyond User Testing

Andrew Swartz of Serco Usability Services discusses ways in which you can ask your users to help improve your products, while still respecting the judgement of your design team.

What did you want to be when you grew up? Astronaut? Teacher? Football star? Diva on Top of the Pops?

Me – I wanted to be Alfred, the butler from Batman (but I suppose that’s mainly something between me and a future psychiatrist). No matter, the point here is that you could ask a thousand kids, and I bet not one would say they wanted to be a ‘usability professional’. Most probably, they wouldn’t know what one is. In fact, most of my friends and family don’t know what I do, even after a dozen painful explanations.

Those of you reading this article on NMK at least are likely to know what usability is, and since you had the good sense to open this article, you may even have secret aspirations to make your products easier to use.
But even a knowledgeable audience may not know that there is a lot more to usability than user testing. That’s what this article is about. It will tell you about a variety of ways you can ask your own users to help improve your products, while still respecting the judgement of your design team.

User testing is a great place to start

Let’s start with the technique you probably already know. A user test is a brilliant research technique to evaluate what users like about your product, and what drives them mad. It can be run in a number of ways, but most often involves bringing five to twelve typical users into a lab with recording and one-way glass facilities. The user conducts normal tasks while thinking aloud, and the observers behind the one-way glass see for themselves the effects of using 9-point pink text on a red background.

Typically it takes a day or two to plan, and another day or two to run. You can hire professionals like us to run it for you, or if cash is tight, you can do it yourself. (We offer superb courses to teach the skills you’ll need, or you can learn about it from books.)

What makes the technique so effective is that even though the number of users you see is small, you will have a strong sense of the main issues even after the first two or three users, and by the tenth user you may actually start getting bored. It’s good value for money.

Academics say that usability testing is all about uncovering usability problems, and it is very good at that indeed. But we think it is about much more. After working with dozens of the biggest names in the industry, I have seen that user tests have a number of surprising beneficial side effects:

• They bring teams together. Even the most argumentative teams are likely to develop a common vision after seeing a dozen users stumble over the same bits.
• They energise projects. There’s nothing like seeing real users getting excited about a new product to let a team know they are working on a winner.
• They teach you about your users’ desires and needs. It can be liberating to know what it is your users actually want, so you can focus efforts on meeting real needs rather than wasting time.
• They answer specific design questions. In the office, a design team can end up in bitter disagreements about whether a button should be black or green, or text should be 10-point or 11-point. Usability tests allow you to make decisions based on real data, not just on guesswork.
• They help you understand local audiences. If you are localising a product, user testing allows you to see that it works with different kinds of users.
• They help teams meet early milestones. A user test requires all the components of a project to work together well enough (even if not perfectly) so a user can work through them. The user test will focus the team so all the components are ready by a specified date.

…but there are many more choices

So that’s usability tests. What other options are there? For that matter, why do there need to be other usability research techniques.

Here’s the thing. User tests give brilliant data, but they often come too late to address big problems. By the time you have a prototype complete enough to include in a user test, you will have spent tens of thousands of pounds on development costs, your marketing team will have a launch date and a sales approach already in mind, and everyone will have become so invested in their ideas that they may find it difficult to let them go.

Over the years, good researchers have developed quick and affordable techniques that work at various times throughout the development cycle, usually to validate and inform decisions just before the cost and risk levels are ready to go up.

Some examples:

• As soon as the idea surfaces. Most projects start with a Big Idea. You can use quick field-based techniques to validate your vision before a single hour of design work is wasted. Once, when working for a major publisher who was developing software for farmers, we determined that their target market was less interested in tracking livestock than in tracking subsidies. Big change, big savings.
• Once the first messy drawings are available. If you already have your Big Idea in hand and you’re starting to work on the overall architecture of your system’s interface, you can validate and refine your ideas with nothing more than rough drawings using a technique called group collaborative design. In a day or two you can iterate the design half a dozen times, and you’ll then have a validated design you can hand to your designers and coders, secure in the knowledge that the fundamental design is sound.
• When you get worried about your competitor. If you work in a competitive marketplace, you may be familiar with the sinking sensation you have whenever your competitor issues a new release and you get your first look to see they’ve just added the feature you’ve been thinking about for ages but had been told was too difficult to implement. Competitive user testing helps you evaluate your competitors’ specific features or general approach so you can determine which features are genuinely appreciated by users, and where their weak points are.

There are plenty of other techniques too – ways of getting users to develop or validate complex information architecture, ways to combine the strengths of a focus group with the insight of a user test, ways to gather feedback over the web or the phone, and ways to assess a system even when there are no users yet.

Three rules

As you think about ways to make your products more usable, consider these three rules:

• Qualitative and cheap is usually better than quantitative and expensive. There may be a temptation to involve 50 or 100 or 150 users in and run a study like a first-year Psychology experiment, and it’s great if you have the time and budget to do that. But you can often get data just as useful – sometimes even more so – from a small group if you use the right techniques. Quantitative techniques are slow and expensive, and often only tell you that a phenomenon exists; in contrast, qualitative techniques like the ones discussed above are often quick and cheap, and tell you why the phenomenon exists.
• Early is better than late. The earlier you can get user feedback, the more cost effective it will be. If you can learn about a major problem before coding begins, you’ll save more money than if you discover the same problem just before you ship.
• Something is better than nothing. The most important lesson is that it’s always better to get some user feedback than none at all. If you’re not doing anything user-centred at all yet, don’t be intimidated into thinking you need a large, complex project run by professionals. You can learn to do many of these techniques on your own. You’ll be amazed at how large an effect you can have on your organisation with even a one-day study run in your own conference rooms.

About the author. Andrew is a Principal Consultant at Serco Usability Services, running commercial research projects and overseeing relations with government clients. He has a particular interest in the effects of high technology on society and democracy. In addition, Andrew manages the group’s commercial research and development activities. Andrew serves as a key instructor for usability and design courses offered by Serco, and information on his next course can be found here.