Activities
Scoping, Secondary Research, User Interviews, Max-Diff
Scoping, Secondary Research, User Interviews, Max-Diff
The organisation provides an Employee Assistance Program (EAP) and digital wellbeing platform to employers. Alongside the employee-facing app, HR and wellbeing leaders use a reporting portal to see how their workforce is engaging with the programs they pay for. That view is how employers judge whether the program is working.
Over time, the portal's reporting had grown to cover five areas: guided wellbeing programs, wellbeing indicator scores, health assessments, challenges, and a rewards and tiers system. Each module carried several metrics, and each metric had a reason to exist. As the design team prepared to rework how this data was presented, the core question was simple: what should an HR leader see first?
Without evidence, the answer would default to whoever argued most persuasively, or to showing everything at equal weight. This study was designed to put a rank order on every live metric, so that layout decisions would rest on what users valued rather than on internal opinion.
Which wellbeing metrics do HR leaders value most, and least, when judging their program, and what should the reporting portal lead with?
I defined success up front as a defensible rank order for every module, an explicit deprioritise list, and recommendations specific enough to apply to a layout without another round of debate.
Rather than asking users to imagine an ideal dashboard, I started with a metric audit across the book of business, listing every wellbeing metric already reported to clients. That gave me 27 real metrics. Grounding the study in metrics that already existed meant every result would map to something on a real screen, and the team could act on it straight away.
When people rate reporting metrics on a scale, almost everything comes back as important, which gives a design team nothing to prioritise. MaxDiff forces a trade-off: in each task, participants choose the single most and single least important item from a set. Repeated across the set, this produces a true relative ranking and shows what people actively don't want, not just what they like. I grouped the 27 metrics into their five reporting modules so that every choice compared like with like (a program metric against other program metrics), with four to ten metrics per module. Each module's question was worded to match its content, for example "most and least valuable wellbeing indicators" and "most and least crucial assessment data points".
I recruited 47 participants. Most (72%) were employer clients, the HR and wellbeing leaders who use the portal, drawn from a range of key regions so the result would not reflect one market's reporting habits. The other 28% were customer success managers, who sit between clients and the product and hear reporting questions first-hand. Including them gave a second view of what clients ask for when they review their program. The survey was in field for two weeks.
For each metric I calculated a preference score: the percentage who chose it as most important minus the percentage who chose it as least important, giving a range from −100 to +100. I then ranked metrics within each module. I deliberately kept the raw best and worst percentages next to every score, because the score alone can hide what is going on. Total wellbeing index score was picked as most important by 92% of participants and as least important by nobody, which is near-unanimous agreement. By contrast, Total programs started scored only +8, but that came from 34% best and 26% worst. It wasn't a metric people ignored; it was one that divided them. That distinction mattered when deciding what to demote and what to keep one click away.
At first, the results read as five separate answers to five separate questions. The useful insight came from stepping back and lining the modules up side by side, comparing what the winners had in common and what the losers had in common, regardless of module. Four patterns held across the modules:
Because MaxDiff measures what people prefer, not why, I kept the scores and my interpretation of the reasons clearly separate in the report. The "why" was presented as reasoned interpretation the team could test, not as fact.
I delivered a research report to the Employer Experience Design team, structured so a designer could act on it module by module. For each module it showed the top three metrics alongside the reporting component each one maps to, so the team could see the finding in the context of the screen they would be changing. Full results tables and my interpretation of the reasons followed. The report closed with three rules for the reporting hierarchy, each backed by a score:
It also included an explicit list of metrics to deprioritise: per-user averages, steps tracked, readiness-to-change charts and distribution of risk. My recommendation was to keep detailed data available for those who want it, but not in prime position.
The most valuable output of a prioritisation study is often the list of things to remove. Knowing that four per-user averages consistently finished last was as useful to the design team as knowing what came first.