Prototype Usability Study
I proposed, designed, recruited, moderated, and synthesized findings for a usability study to identify usability issues and opportunities in two potential prototypes for a B2B SaaS analytics software with the goal of providing actionable recommendations to move forward with the product.
Solo Researcher
Role
B2B SaaS
Industry
Recommendations Adopted, Prototype Advanced
Outcome
The Problem
A software team had developed two distinct prototypes for the same business analytics product and needed a decisive, evidence‑based recommendation on which approach to carry into beta.
Getting Started
To begin, I created a research plan that would gather data about each prototype and evaluate them across core jobs-to-be-done. The study needed to capture behavior and perceptions of each user, and use them to surface insights about the prototypes' usability.
Study Design
I set up moderated user testing sessions for each prototype, using the think-aloud protocol to complete 5 tasks that directly correlated to the necessary jobs-to-be-done for the product. The test focused on identifying usability issues in order to improve the product. After each task, I gathered SEQ data and administered the SUS survey at the end of the session. Sessions were recorded and time-stamped to correlate comments with events.
Recruitment
I recruited 8 testers for each prototype (16 total), balancing familiarity with similar roles/products and technical literacy to reflect a realistic range of experience that was even across both prototypes.
What Data was Captured
Single Ease Question (SEQ) per task
System Usability Scale (SUS) post-test
Time-on-task & completion rates
Critical incident logging
Qualitative theming & affinity mapping
Think-aloud protocol recordings
What the Data Showed
Both prototypes exhibited major usability deficiencies; a lack of affordances and in-app guidance left users confused about what information was interactable and what was static.
Prototype A encountered issues with data table presentation, with users desiring more control over the data visualizations.
Prototype B showed lower usability in most observed areas, but saw much higher usability in the interactive AI system.
Testers of both prototypes expressed a desire for a deeper level of drill-down data than currently available, and frustration with navigation and non-clickable data.
My Recommendations
After synthesizing the data from both studies, I delivered a set of actionable, evidence-based recommendations that directly informed the product team’s next steps. This included which prototype I suggested the team move forward with, as well as what changes were needed to improve the prototype's usability and alignment with user needs.
“This is amazing. I have so much to go off of now, I’ve already got the team working on all of these findings.”
— Prototype Product Owner
Outcomes
The product team adopted my recommendations, moving Prototype A forward to beta after implementing the proposed interaction and grid updates. Additionally, they ported the AI features from Prototype B, which my research showed to be more intuitive and trustworthy from a user standpoint.
This test also served to align the executive stakeholders toward a data-backed path forward, reducing debate and accelerating the roadmap.
This outcome demonstrated how structured usability testing and evidence-driven recommendations can shape product direction, resulting in a hybrid solution that combined the strongest aspects of each prototype and moved the product on to the next stage with full stakeholder alignment.
