Usability Testing: Recruiting a Sample of Users

Usability testing involves a sample of test users. These users should possess characteristics that are found in your eventual buyers, users, and/or visitors. Many products and services have a variety of different user types or groups, so it is important to recruit a range of test users to tap into these variations. Ideally, time and budget permitting, when dealing with a product with multiple user groups, you should test them separately to ascertain how they perform on certain tasks and how they feel about the product features specific to their particular group.

Companies and businesses will often farm the job of test user recruitment out to a specialist firm or group, but they will also need to take demographics and other qualities and characteristics into consideration when selecting potential participants on their behalf. This requires knowledge of the target users and the nature of the product under investigation, so the recruitment team will need to gather information before recruitment begins.

Where should you recruit participants?

On occasion, usability tests may be carried out with employees from the related business, company or group, but this is generally not recommended due to the potential for bias. If the employee feels they must say positive things to keep their job, the testing sessions will probably fail to yield the desired results. However, it is common for the design team to test the prototypes or early versions themselves or with a few employees from another department at an early point in the process, but this shouldn’t be in place of proper rigorous and impartial tests.

In some projects, recruitment is a straightforward process, as there are a ready supply of people willing and able to take part, but when the product has a very specific user group recruitment must be targeted at identifying the best possible candidates from the available pool. For example, when testing a new mobile app for checking which supermarket offers the cheapest prices on specific products, recruitment should be relatively straightforward, as the target users would be the general public. Recruiters could then simply ask people outside supermarkets if they would like to participate in the study.

In contrast, if you are developing a new and complex system for a specific group of workers in a company, you would need to either take a sample of these eventual users or identify the skills and responsibilities necessary for this role and try to find an equivalent group from another source.

Screening Potential Test Users

To determine potential participants’ suitability for testing, they will be screened, which involves asking questions that help the recruiters establish whether they are representative of their target users.

Screening questions may be very general, such as ‘how old are you?’, ‘where are you from?’, and ‘where do you buy your groceries?’, or specific to the product in development, such as ‘do you have any experience using online click-and-collect services?’, ‘how frequently do you use handheld computer consoles?’, or ‘what is your preferred method of purchasing electrical goods?’.

(Go to screener templates at usability.gov for Mobile Device Testing and Website usability test screening documents)

The Costs of Recruitment

The chosen test users will usually receive some ‘reward’ or financial compensation for their involvement in the usability testing sessions. These will vary according to the amount of time the testing takes and, sometimes, the complexity of the tasks. If the users are recruited from a specific group of skilled professionals, the rewards may be higher in order to compensate them appropriately for their time. Rewards include payment (e.g. $5, $10, or maybe more depending on the factors mentioned above), gifts (e.g. vouchers and products), and expenses (e.g. travel, food, and hotel costs). If you do offer compensation, you must keep documentation for your accounting and/or personnel department to show the costs of the process. Documentation of this type is typically in the form of a signed receipt from the test user(s), which has the added benefit of showing that they have received their compensation.

If the company or business uses a recruitment firm, the cost of compensating test users will be eclipsed by their costs. Recruitment groups or individual recruitment agents are typically paid a fee for every test user that fulfils all of the testing requirements (i.e. fits the specified demographic and successfully completes the testing session).

The recruitment process should be a relatively painless procedure, but it is important to provide the recruiter(s) with sufficient information to choose the right people for your project. This shouldn’t necessarily be an expensive process either, as people are often willing to participate for very little and a small sample of around 5 to 7 is usually enough.

Links

Recruiting Usability Test Participants (Usability.gov)

Ethnio: A site for recruiting users

How to find great participants for your user study

Source: IDF