COVID-19 Antigen Test
At-home nasal swab test kits for the novel coronavirus.
CLIENT
Becton Dickinson (BD)
TEAM
Scanwell Health (YC 18)
-
2 product designers
-
2 UX researchers
-
1 Head of Design (direct manager)
This only lists design team. Contact me for full team details!
CATEGORY
Product features:
-
iOS/Android for mobile
-
Physical nasal kit
-
Physical & digital instructions
-
B2B2C, B2C, 0 to 1
DURATION
1.5 years (Sept 2020-Jan 2022)
Launched in Oct 2021
THE CHALLENGE
Scanwell Health collaborated with Becton Dickinson (BD) to create the first digitally read at-home COVID-19 test. This was the first time BD and Scanwell were partnering together to create a product and with that came its own set of challenges, such as trying to understand the best way to communicate, how each team functioned, and the varying work styles of each team. One major challenge we faced continuously was adapting and pivoting our product design to the rapidly changing customer segments, user needs, and FDA requirements and expectations.
THE GOAL
Build a COVID-19 Antigen test that employees can perform at home, receive instant results, and easily share these verifiable results with their organizations.
Our original target market was for employees to share results with their employers (B2B2C). This initial scope significantly impacted the eventual design of the product especially later on when our product became available for the general public.
TLDR; FOR THE BUSY SOULS
MY ROLE
I led the UX research team. I was the main UXR contact and advocate with stakeholders. In the first third of the project, I was the main UX contact until more designers joined the team.
BUSINESS CONTRIBUTION
Helped over a million users successfully perform the test in the span of 3 months (less than 2% usability fail rate). Produced accurate data on expected failure rate 4 months prior to launch which prevented delays in prod dev timeline.
BEST LESSON
For products that are addressing national emergencies, it is advantageous to time the market because user perceptions, desires, and motivations shift significantly at a wide range (although hindsight is 2020).
MOST SURPRISING INSIGHT
Throughout all 11 usability studies, users who failed to perform the test correctly had a higher confidence level than users who performed the test correctly.
THE PROCESS
PARTNERSHIP W/BD
The various teams involved in this project worked concurrently with daily syncs. My team at Scanwell led the efforts on everything that involved the mobile app and the overall user experience of the product. The BD team was responsible for the marketing, manufacturing, and distribution strategy.
Aside from the clinical study which was run by BD, all other usability studies were run and led by me and our UXR team.
11
usability studies
300+
users interviewed
20,000
data points
8
competitive analysis reports
UXR PROCESS
STUDY TIMELINE
STUDY INTRODUCTION
Product managers, business dev, and/or BD would typically request answers on certain features and flows. Some studies were initiated by design team.
RESEARCH PLAN + INTERVIEW SCRIPT
UXR team would break down all aspects of the study in a google doc and make adjustments based on feedback.
PROTOTYPE + RECRUIT USERS + SHIP KITS
Logistics, logistics, logistics.
INTERVIEW + REPORT
Mid-way reports were common for faster data synthesization.
MY ROLE
EVOLVING FROM SOLO DESIGNER T0 UXR LEAD
User research and usability testing were my primary focuses in this project; however, during the first third of the project, I had a more general UX role- in charge of all things interaction design and instruction design. As we brought on more team members to the design team, I was able to hone in on our research processes and dive deeper into the major potential risks in our product.
UX Designer
Protocol creator
Recruiter
Interview Moderator
Notetaker
Logistics Lead (i.e. BFFL with FEDEX employees)
Presenter
User Advocate
Deck maker
Mentor
INITIAL CHALLENGE DURING THIS PROJECT:
HOW MIGHT WE ALLOW EMPLOYEES TO SHARE ACCURATE AT-HOME COVID-19 ANTIGEN RESULTS WITH THEIR ORGANIZATION?
THE CHALLENGES
THE INITIAL PROBLEMS
Our goal was to create a test kit and digital app that would allow users to perform an antigen test in their homes and receive results the same day. As it was the first digitally read, at-home COVID-19 Antigen test, many challenges existed.
CARETAKER USER SEGMENT
The original target market widened to a more general population to accommodate for shifting user needs during the pandemic, bringing the additional challenge of designing for scenarios where adults may need to perform the test for children and older users.
REGULATORY RULES
Replicating real-life testing environments with remote users was difficult due to IRB rules regarding the collection of bodily samples during studies. Changes post-FDA authorization also had to be thoroughly analyzed to contemplate if they were too drastically different from the original design.
TESTING PHYSICAL & DIGITAL
As this product required a user to perform the physical test with digital components (scanning and instructions), we had to figure out the best remote strategy for testing all components concurrently during the peak of the pandemic.
OH WAIT,
THERE'S MORE
Our intial goals changed and expanded as FDA refined and altered the requirements for this test with growing data and understanding on the workings of this new virus.
B2B2C + B2C
We anticipated that organizations would purchase large bulks of these tests to allow their employees to perform the test prior to coming back to work; however, the user segment expanded when it was decided that the test would be available for anyone to purchase through Amazon. The change in distribution tactic caused device compatibility to become a big pain point.
WEEKLY REPEAT TESTING
We did not foresee how long the pandemic would last in the states, and we assumed that the test would be for one-time use. As the pandemic extended past year 1 to 2, the length and design of the flow had to be significantly updated to accommodate repeat users as that feature became a common and essential value proposition to users in the US.
2 TESTS PER KIT
The packaging and visual assets were designed for one test per box. Post-submission to the FDA, we learned that the FDA now wanted serial testing in every COVID-19 test. This meant we had to quickly figure out the most seamless and least complex method of including 2 tests in each kit without significantly altering the packaging design, visual assets, and user flow in 2 months.
USER JOURNEY
GUARANTEED FEATURES
All of the diagnostic tests at Scanwell have "universal features" in the user experience as I like to term it. This was also the case for the antigen test kit.
01
PHYSICAL & DIGITAL
User receives test kit and downloads our app
02
TEST IDENTIFICATION
App identifies test type
03
SELF-TESTING
User performs test while following app instructions
04
SCANNING
User scans the test strip/cassette with scan card
05
RESULTS & NEXT STEPS
User receives test results
THE APPROACH
EXPERIMENT. ASSESS. GO.
We had many unpredictable obstacles that forced us to alter our approach. To handle the unexpected, our main strategy was to constantly and rapidly innovate and experiment with various programs, tools, and processes that could increase the efficiency in how we tested concepts, collected data, and shared data.
CONSTRAINT
APPROACH*
FREQUENT TIME-CONSUMING UPDATES TO STAKEHOLDERS
CONSOLIDATED WIREFRAMES ON FIGMA AND GOOGLE SLIDES
TECHNICAL LIMITATIONS FOR STUDY PROTOTYPES
EXPERIMENTED WITH ORIGAMI, TESTFLIGHT, AND PROTOPIE
NOT ENOUGH MAN POWER FOR DESIGN AND RESEARCH
HIRED 2 FTE FOR DESIGN AND RESEARCH TEAM
*Not all of our approaches were perfect solutions. To hear more details, contact me!
UNDERSTANDING THE USER'S MENTAL MODEL
Another essential point we had to constantly question was the design of the studies themselves. We had to put ourselves in the shoes of our users.
How do we capture enough data about a user's home environment to understand their scanning struggles before it becomes too invasive?
Would hosting a remote study through zoom deter a user from naturally moving away from the camera to get a better scan?
In what mentality would users actually be using the test? Do they have to think that they show signs of COVID-19? In contact with someone positive? How long ago?
THE FRAMEWORK
TRIANGLE DEFENCE
We had three shields to our strategy:
HYBRID USABILITY STUDIES
We ran usability studies that combined both quantitative (statistical data on task analysis) and qualitative (the why and how data) aspects of user research. It was a true mix between summative and formative usability methods.
WEEKLY TESTING PROGRAM
Post-market launch, our reliance on live data for improvements decreased the velocity and depth of our insights. For quicker data retrieval, I launched the Weekly Testing Program (WTP) where we ran highly focused and small usability interviews on a weekly basis (e.g. repeat users, new concepts).
01
02
03
COMPETITIVE ANALYSIS
The market was shifting rapidly and the number of high priority tasks increased exponentially. To ensure that we still had a thorough understanding of the landscape, I launched and led weekly 2 hour deep dives on competitor's test kits with design team members.
01
HYBRID USABILITY STUDIES
This mixed-methods approach was our main defender and most go-to tactic. Remote interviews were structured so that the first half would consist of observing users performing the test and taking note of which tasks users failed while the second half was deep dive questions into the why behind the users' actions.
The combination of different data allowed us to not only get success rates and the potential percentages on how many users would behave in a certain way, but also the possible reasons for users' reactions.
Studies ranged anywhere from 15 to 30 users, 30 to 60 minutes, and lasted 2 to 3 weeks to complete the whole study from creating the research plan to sharing the results with stakeholders.
02
WEEKLY TESTING PROGRAM (WTP)
Relying on solely Defence Shield 1 slowed down our ability to validate new ideas and concepts as the list of priority items expanded. To conquer this, we launched weekly "mini" studies in Q4 2021 that allowed us to quickly gauge which ideas and new wireframes would be potentially high-risk changes.
I proposed that we stagger our studies so that we could obtain more value from each user since we were spending a significant amount of time and money to recruit participants, ship test kits, and run these interviews. One method was to have first-time users intentionally save the extra kit they obtained for a follow-up study in a couple of weeks. This allowed us to gain even more insights into not just first-time users but also serial testers (2 user segments in one!).
03
COMPETITIVE ANALYSIS
Usability studies weren't enough to create a strong defensive strategy. Every other week we were hearing about a new competitor, and we were playing catch-up trying to keep up with the industry's intense pace in product development.
To prevent the team from getting overwhelmed, I set up 2-hour weekly competitive analysis sessions with design and research team members to run deep dives on products that had already been on our product team's radar.
We analyzed end-to-end user journies, marketing strategies, product pain points, manufacturing quality, instructions design, and onboarding/offboarding experience to formalize insights on what we should and shouldn't do.
Reports were then shared with the rest of the team through Slack, Threads, and Clickup.
HIGH LEVEL INSIGHTS
FOR MARKET LAUNCH FACTOR IN CHANGES IN USER PERCEPTION AND MOTIVATIONS
SCANNING DATA NOTORIOUSLY DIFFICULT TO CAPTURE REMOTELY
VIDEO QUALITY AND FIDELITY MORE IMPACTFUL THAN WARNINGS AND COPY
USERS WHO FAILED ARE MORE CONFIDENT IN THEIR PERFORMANCE THAN THOSE WHO PASSED
DISCOVERY PHASE SIMPLIFIED
INSIGHTS CATEGORIZED 1, 2, 3!
There were three major data buckets that we were primarily focused on throughout this project.
01
TASK PERFORMANCE
Our first and primary objective was ensuring that our instructions were clear enough that users would correctly perform the actions needed to complete the test. Task analysis through observational studies were the best plan of attack to capture this type of data. Nearly all of our usability studies included task analysis.
One of the earlier task analysis flows I created
02
SCANNING
PERFORMANCE
Equally as important as performing the test correctly was ensuring that users could scan the test stick correctly so that we could analyze their results. We ran numerous internal studies at small and large scales to ensure that we were capturing enough data on the major ways a user could fail to scan. In 2021 Q4 I thought we needed to perform in-person scanning studies for more accurate data and created a complete research plan and proposal. For more information, contact me!
Internal scanning study with BD employees
03
RESULTS
COMPREHENSION
One of our main value propositions revolved around being able to send reliable test results to organizations. At the same time, we had to ensure that our results content was aligned with FDA regulations. We juggled and debated on how to share dense information, ensure users understood their next steps, and create a clear navigational system to share results.
Earlier iterations of the results screens
ONE THING I WOULD DO DIFFERENTLY
I regret not pushing harder for more clarity on which user segment we were targeting at the initial market launch. Relative to this, I regret not pushing for more onboarding data prior to our launch.
Both aspects killed our app store and Amazon reviews because we had not accounted for device compatibility issues with the general public. Since we had assumed we were going to ultimately be designing for employees, we had a specific set of phones adapted to our camera scanning capabilities. We strategically tested our algorithm on the latest and most common smart phones since it would've been impossible for us to analyze every single smart phone in existence.
"I really like this app. [These are] better results to send to an employer because otherwise you just send a picture of the test stick to your boss."
WTP user_Week 4
MAJOR INSIGHTS
HIGH-RISK PAIN POINTS ANALYZED
There were two major areas that needed long-term testing and numerous iterations to get right. By the end of Jan 2022, there was still room for improvement in both features.
PP1: LET ME SCAN IN PEACE!
The initial major pain points revolving scanning included:
LONG PROCESSING TIME (15 MIN) + SHORT SCANNING WINDOW (5 MIN)
Users only have 5 minutes to successfully scan their test cassette and scan card once it has finished processing for 15 minutes. This increases the chance a user may forget the correct way to scan and troubleshoot.
DIFFERENT + MORE COMPLEX COMPARED TO COMMON 2D SCANNING PRODUCTS
Unlike most scanning products (e.g. mobile bank check deposits or PDF scanning apps), our scanning experience required users to scan a 3D object that was sensitive to colors and had a very specific time limit.
CHANGES IN NATURAL LIGHTING + LOCATION OF TEST KIT ITEMS
Since the test takes roughly 20 minutes to complete before reaching the final 5-minute scanning step and users are interacting with many items, there was a high chance that something may have moved out of place from when they first started the test and/or that the lighting conditions had become worse while the test was processing.
Older wireframe iterations of the HLC instructions
Early storyboards of a "How to scan" video I created
Quick sketch mockups I created during a brainstorm on what the best instructional diagram could look like
THE BIRTH OF THE HOME LIGHT CHECK
Instructing the users on how to scan effectively right before the 5 minute final capture scan wasn't producing the results we needed. It was too last-minute, there was too much brand new information a user had to absorb and retain in an already time-sensitive, stressful environment.
That was how the Home Light Check (HLC) was born. Prior to starting the actual test, a user would read static images and copy that explained how to scan correctly.
They would then attempt to scan just the scan card in the Home Light Check that emulated a close situation to what they would be experiencing during the Final Capture step.
Our hope was that this would be close enough to the last scan that users would have figured out all the issues and variables that could positively and negatively impact their scanning experience.
PP2: IT TAKES TOO LONG TO TAKE THE TEST MULTIPLE TIMES!
Serial testing was not something in our horizon when we initially submitted our application to the FDA for EUA (Emergency Use Authorization); however, based on many of our studies post-launch, we realized this would be a critical feature.
Repeat users were becoming impatient with the mandatory videos in each step of the test. We had originally decided to make all videos mandatory so that we were absolutely confident that users were at least hearing or seeing the required instructions that were set by the FDA.
As the pandemic transitioned from a temporary blip in our timeline to a potential endemic, we began to shift our focus to the needs of repeat users.
SUPER COOL UPDATE TO THIS EXPERIENCE BUT CANNOT YET DISCLOSE :(
I cannot disclose the details on how we changed this feature as it has not yet launched as of Jan 2022.
For more information, contact me!
"On my first day at Scanwell, I witnessed Michelle running 30 back-to-back interviews to ensure we had enough Android users represented in our study.
All in 2 days."
Katherine Hsiao
UX Researcher at Scanwell Health
THE RESULTS
Conclusion by the #s
As with all early-stage start-ups, we didn't get it perfect the first time around but we did make significant improvements throughout the year we built this product. I'm especially proud of our high usability percentages.
21.4%
improvement in scan success rate since the introduction of the Home Light Check
12.2%
7.7 to 8.92 usability rating improvement from first to last usability study
42%
51.6% successfully performed the test during the first usability study. 94% succeeded by last usability study
After a year working with BD as a major stakeholder on this project, BD acquired Scanwell Health at the start of 2022
Lessons Learned
NOT HEALTHY NOR SUSTAINABLE TO RUN 30 INTERVIEWS IN 2 DAYS
NEED VARIOUS PROTOTYPING STRATEGIES FOR MATURE PRODUCTS W/OVER 1 MILLION USERS
I TEND TO STICK TO STATUS QUO WHEN I'M BURNED OUT INSTEAD OF TAKING THE INITIATIVE
MARKETING PLAYS A HUGE ROLE IN B2C PRODUCTS