⬅ Go back to Illumina internship

BaseSpace Sequence Hub ver 5.0 Usability Testing

I worked as a lead researcher for this project, leading all efforts in defining the research opportunities & goals, designing the research study, identifying, recruiting, and scheduling study participants, gathering and analyzing user feedback, and communicating the results in written/verbal format to the stakeholders (Product Managers, UX Designers, Software Engineers).

Define & Hypothesize

Project Background

When I joined Illumina, BaseSpace Sequence Hub (or BSSH) was going through a big revamp from version 4.0 to 5.0, which meant it was going to see a lot of new features and additional functions that would help users manage, share, and analyze their genomics data more effectively and efficiently. It was also expected to see UI changes, including new layouts and different dropdown menus.

The stakeholders (PMs and Engineers) wanted to understand if users:

1. Could find and utilize the new functions/menus

2. Would appreciate the upcoming changes

3. Had any concerns or trouble with these changes. 

Research Objectives

1. Examine the usability and usefulness of the major new features of BSSH 5.0.

• Can they find the new features?

• Can they perform the new features?

• Do they see themselves using the new features?

2. Understand current users’ thoughts/concerns regarding the new version.

• What concerns do they have and why? Positive or negative?

Design & Plan

Research Design + Learning Plan

With the research objectives in mind, I decided on gather user feedback by conducting task-based usability testing. Each 1-on-1 session would be one hour long, and would have the participant to try out the key new features of BSSH 5.0 based on some scenarios.

Since BSSH 5.0 was going to see a lot of fundamental changes not only in the terminology/concepts but also in its functionalities, I worked with my stakeholders to first scope down the research focus to few major changes we want to initially test with the participants. (For example, we decided to not to gather feedback on API-related new features that would require prior coding knowledge for users to set up and utilize, but focus on features that are readily available via the software UI.)

The first 5 minutes of the testing would be to warm up the participants, and gather background context in how they currently use, store, and manage their data on BSSH in their lab environment. 

Next step would be task-based usability testing, which would be testing newly launching features based on given scenarios (e.g. "Let's imagine that one of your biosamples is stuck in analysis. How would you find it and requeue the analysis?").

The last 5 minutes were reserved to wrap up the participant's experience with BSSH 5.0 based on SUS (system usability scale), and gather any final feedback they have (open-ended).

Participant Recruiting

I focused on recruiting high-throughput users of BSSH 4.0 who were already familiar with most of the major features (e.g., setting up, monitoring, and QCing their runs and using the applications for data analysis). This was to make sure I was getting participants who had an actively working knowledge of BSSH and what it offered, so that we would be able to gather as accurate user reactions as possible.

It was a challenge recruiting participants for this study, since the users I was looking for would be those who worked in specific lab environments with highly specialized working knowledge. Therefore, I worked very closely with Product Managers, Inside Sales department (Territory Account Managers & Regional Account Managers), and Tech Support department (Technical Application Scientists) to identify good candidates for this testing. I also reached out to past participant contacts, Illumina Accelerator program, and other software user groups such as LIMS Luminaries, to find potential participants.

As a result, I was able to recruit 3 external and 4 internal users who fit our participant criteria. I excluded developers and bioinformaticians for this study since this study did not focus on API-related features.

After realizing the inherent difficulty in finding participants for user studies related to BSSH, I worked with a Business Analyst on my team to gather the contact list of internal users of our product. I then created a user group on the company's internal communication platform, which could later serve as an active channel to spread word on upcoming user studies and help find appropriate candidates. In addition, I brainstormed ways to foster this group to reach its full potential as an active user forum.

Conduct & Iterate

Study Logistics

After the usability testing plan was shared with the stakeholders to receive final feedback and thoughts, 1-on-1 sessions were conducted over the next 2-week period of time.

First, I provided participants with a test user account to BSSH 5.0, where I have previously set up dummy data that could be used for test scenarios. The participants shared their computer screens via WebEx platform, and their screens and voices were recorded under their permission. I conducted all sessions via UserZoom (I leveraged the platform but moderated the session), and decided whether it would be done in-person or remotely based on their geographical location. I introduced the Concurrent Think-Aloud (CTA) protocol to the participants and encouraged them to freely share their thoughts as they went.

Data Collection

I took notes via laptop during the sessions, which were saved real-time on the team's user research database on Confluence. Quantitative metrics such as success rate were also gathered during the testing. The screen recordings were also later reviewed to fill any gaps in the notes. The test design and the script also continued to see minor iterations based on feedback from the completed sessions.

⬅ Go back to Illumina internship