Recruiting participants with both experience and no prior exposure to hybrid-working platforms proved to be a challenge.

Experienced users provided valuable insights into existing mental models, including which features were intuitive, which felt cumbersome, and what functionality they deemed essential or missing.

On the other hand, including participants without prior experience was crucial to evaluating the platform’s ease of use for first-time users. Balancing these perspectives was essential but difficult to achieve.

Preventing observer bias and ensuring results remained authentic required careful moderation. This involved asking neutral, non-leading questions, avoiding offering assistance during usability challenges, and allowing participants to navigate tasks independently.

Managing the Hawthorne effect, where participants alter their behaviour because they know they’re being observed, added complexity to maintaining objective results.

Ensuring the testing environment was technically sound posed multiple challenges. It was critical to capture clear audio of both participant and moderator voices, record screen interactions, and observe facial expressions for non-verbal cues.

Remote usability tests introduced additional difficulties, such as software and hardware compatibility issues, internet disruptions, and prototype performance limitations, all of which required proactive troubleshooting to avoid interruptions.