Planning your recruiting needs

In order to be successful, one must consider several aspects of the recruiting criteria in order to get the correct participants. Above all else make sure you know your user. If you’ve been involved and have used a human-centered approach this shouldn’t be too challenging. You’ll want to know who they are, what they do, and how the system is intended to help the user and provide utility to the user.

Knowing your user will help you recruit for your test. You’ll need a great recruitment plan whether you have existing users or target users of a newly developing system. Knowing your user is vital. These users will represent the actual users and can provide informative and valid feedback that you will need to make meaningful design decisions.

Understanding your users

Meet as early as possible with stakeholders. Make sure you’ve prepared topics and questions in advance when meeting with stakeholders. Here are some example stakeholders that you will want to interview; project owners, business analysts, developers, marketing department, quality analysts, business process SMEs (subject matter experts). There will be times when you’ll need to improvise during the actual interviews with stakeholders, but make sure you’re prepared to limit the improvisation.

Be sure to understand the goals of the study. This should be a topic you cover with stakeholders as a part of the early interviews. Work with the team to understand which users should be included in the user study. You should also look at the range of users or the number of users that you will need to recruit for a study based on the scope of what is being tested.

Be sure to include the system team in the interviews. While they might not be the best set of representatives or fully understand how users are interacting with the system, they are experts in what the system is supposed to do, and can also help with the intended user information.

Here are some examples of what topics to cover during the interviews.

  • Job title and descriptions of stakeholders
  • Job role
  • Do they use the system and how often
  • Most critical tasks and order in which they complete them
  • Where is the system used
  • How complex are the tasks
  • Any prior experience with related systems

 

Now that you understand who your users are you are ready to start recruiting. The first thing to determine is how many participants to include in the study. It is best to have at least 5 users to test with, but if you are looking to provide any sort of statistical significance, plan on testing with 12 to 15 users.

While 5 users will most likely find any large issues with the system this small of a sample size will not be recognized for being statistically significant. However, if you only have 4 or 5 users for a particular test don’t refrain from continuing as these users will provide significant qualitative information and feedback.

Prior to any tests make sure you conduct a dry-run. You’ll want to do this for any new study and for any changes you make to your prototype or test script. This ensures your test will run smoothly and you won’t waste any time managing changes on the fly. You can recruit a participant for the pilot test, but I’ve always just used someone from the project team to ensure things will go smoothly for the actual study.

Also, make sure you conduct the pilot test at least 24 hours before the actual testing session. You may find issues with the study materials or prototype that will need to be addressed and this could take some time.

Depending on the system being evaluated, you may or may not need to create a screener. I’ve traditionally set up usability tests for internal, enterprise systems and limit my time doing this step. I provide a guide to the business managers on the type of participants I’d like to include in the test but don’t typically need to spend a lot of time on this part of the setup.

If you’re testing a website that is consumer-facing you’ll want to spend time making sure you’ve prepared a participant screener that matches your main user persona.

How and when to provide incentives

If you have internal users and they are a part of a worker’s union, make sure you check with union officials before moving forward. I’ve run into this on a couple occasions. However, in both cases, the union officials were more than accommodating in finding volunteers for the study.

Whether or not the participants are a part of a worker’s union it’s important to determine the appropriate incentives. Check with management about existing motivators that you can leverage when dealing with internal users. They may have processes in place that allow you to use their existing incentives, making it easier to recruit and conduct the test.

When determining paid incentives, be as generous as your budget will allow and DO NOT neglect to give some sort of incentive. Another important thing to remember is to thank each user. Do not let the incentive be the final thank you. Treat your participants to a verbal, “Thank you very much for your time. Your feedback is extremely important and has been very helpful.” Make sure you say this whether or not you feel that the results of a particular participant provided value.

In Summary

Plan for your study by understanding your users and their needs. What are they tasked to complete based on their use of the system being evaluated? Be sure to determine the best number of participants for your study and incentivize them for their time. Also, make sure you properly thank each user whether or not they provide the feedback you were looking for.

Hopefully this article helps you prepare for your next study! Thanks for taking the time to read the article.