Usability Benchmarking: Why It Matters and How to Do It Effectively

Efficiency

How quickly users can complete a task (e.g., time taken to complete a checkout process).

Effectiveness

How accurately users can complete tasks (e.g., error rate, success rate).

Satisfaction

How users feel about the product (e.g., user satisfaction surveys or Net Promoter Score).

Learnability

How easily new users can perform tasks the first time they use the product.

Memorability

How easily users can recall how to use the product after a period of non-use.

Competitive Benchmarking

Compare your product’s usability to that of competitors or industry leaders. This involves identifying key competitors and analysing their products against the same set of usability criteria.

Analyse the task success rate and user satisfaction between your mobile app and a top competitor’s app.

Internal Benchmarking

Compare different versions of your product or compare usability within your own product over time. This helps measure improvements or regressions in usability after changes or updates.

Evaluate how a new design or feature in your app affects usability by comparing it to previous iterations.

Industry Standard Benchmarking

Compare the usability of your product to industry-specific usability guidelines or standards (e.g., WCAG for accessibility, or ISO 9241 for usability standards).

Assess how your product’s accessibility features measure up to WCAG guidelines.

Usability Testing Tools

Look-back, UserTesting, Maze, Optimal Workshop (for task-based testing and gathering user feedback).

Heat-maps

Hotjar, Crazy Egg (to track user interactions and identify which areas of the page get the most attention).

Surveys & Questionnaires

Google Forms, SurveyMonkey, Qualtrics (to gather user opinions).

Analytics

Google Analytics, Mixpanel, Amplitude (to track user behaviour and conversions).

Benchmarking Reports

Nielsen Norman Group’s Usability Benchmark Reports, Forrester Research, or industry-specific research papers.

Task-based Testing

Assign typical tasks to users (e.g., “Complete an online purchase”) and track metrics such as completion time, success rate, and error rate.

Track the time it takes a user to find a specific product and check out.

Surveys and Questionnaires

Use tools like SUS, NPS, or custom surveys to gather subjective insights about users’ satisfaction.

Send a survey after testing that asks users to rate their overall experience.

Heatmaps / Clickmaps

Track user interactions to visualise where users are engaging on the page, which can help identify areas of confusion or strong points in the interface.

Heatmap data can reveal that users consistently click on a non-interactive image, indicating confusion.

Contextual Inquiry

Observe users interacting with your product in their natural environment to understand real-world behaviour.

Conduct field studies to see how users navigate your app in their daily routine.

Select Participants

Choose a diverse group of participants that reflects your target audience (demographics, behaviours, user scenarios).

If you’re testing a shopping app, select users who frequently shop online, across a range of demographics.

Perform Testing

Assign tasks, track performance, and collect feedback through surveys, interviews, or direct observation.

Ask participants to complete a series of tasks, such as purchasing an item and navigating the checkout process.

Analyse Data

Gather both qualitative and quantitative data, including completion rates, task times, error rates, and user sentiments.

If users consistently fail to find the checkout button, this indicates a potential usability issue.

Present Findings

Summarise the results in a clear, actionable format. Use visuals like charts, graphs, and heatmaps to support your conclusions.

Create a presentation with task completion times compared across competitors and your product.

Identify Usability Gaps

Highlight areas where the product is underperforming. For example, if users fail to complete key tasks, those tasks should be flagged for improvement.

If users are confused about the login process, make it a priority area for redesign.

Recommend Improvements

Provide clear and actionable design changes that address usability issues identified during testing.

Suggest improving the button size or placement for better task completion rates.

Prioritise Improvements

Rank recommendations based on their impact on usability and business objectives.

Rank recommendations based on their impact on usability and business objectives.

Monitor & Iterate

After implementing changes, continue testing and revising to improve usability over time.

Re-test after design changes to see if the task completion rate improves.

Objective Insights

Provides data-driven insights to make informed design decisions.

Understanding user struggles can guide design changes that better align with user needs.

Improved User Experience

Identifying pain points and areas for improvement ensures a seamless and satisfying experience

Reducing task time and minimising errors leads to happier, more satisfied users.

Competitive Advantage

Benchmarking helps you understand where your product stands against competitors, allowing you to refine and improve.

By outperforming competitors on usability metrics, you can position your product as a leader in the market.

Informed Prioritisation

Benchmarking helps prioritise which usability issues to address based on their impact.

Focus on improving navigation if users frequently abandon tasks due to confusion.

Continuous Improvement

Regular benchmarking creates a feedback loop that helps your product continuously evolve.

By iterating and improving based on data, you ensure your product keeps up with changing user needs and expectations.