Architecture Tests: How to Enforce Architectural Rules Directly in Code
Architectural rules that exist only in documentation will sooner or later be broken. It's not a question of if, but when.
Measuring usability of software applications
Today, software is all around us. Often, we do not even notice it; at the same time, however, there are many applications that have a user interface and that we, as users, actively interact with. Whether it is a favorite social network, a music player in a car, accounting software, or an application for controlling a microscope, unless we pack up and leave for a while to a place without mobile network coverage and access to electricity, we encounter software daily.
As the number of applications continues to grow, along with the amount and diversity of their users, the need to design user interfaces that are comfortable and easy to use is also constantly increasing. For application creators, such efforts can attract more users or increase sales; for users, they can save time, improve efficiency, or simply help preserve a good mood.
What Is Usability and Can It Be Measured?
Usability is a concept from the field of UX (user experience) and is defined by the ISO 9241 standard as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a given context of use.
From this definition, it is clear that usability itself is a rather subjective characteristic. Opinions on whether an application is usable, or how usable it is, are always strongly influenced by who is using it. Users may have different levels of education, varying experience with computers, mobile phones, or other devices, different goals, different cultural backgrounds, or various disadvantages compared to others. All of this complicates usability testing of applications, but some options still exist.
Qualitative testing usually involves a smaller number of users—typically just a few people. Ideally, testing takes place in person. We talk with the user and assign tasks they are supposed to complete using the application (for example, in the case of a mobile bank app, we might assign the task: “Transfer 300 CZK from your personal account to account number X with today’s due date.”). While users are performing the tasks, we observe what they do, whether they proceed as expected, where they make mistakes, whether they find everything they need to complete the task, and so on. At the same time, we encourage them to think out loud so that we understand what motivates their actions. From these sessions, we gain important insights into how users work with the application—what they can handle, what causes them trouble, what they like, and what they dislike. However, this type of testing is time-consuming.
Quantitative testing allows us to involve many more users—dozens, hundreds, or even all users. It can take place online without our direct presence. We use built-in measurement mechanisms within the application or existing tools designed for this purpose. The output is numerical data that can help reveal problematic areas in the application.
To measure usability, we must consider all three of its components as described in the definition above. Quantitative testing can be conducted separately or even as part of sessions during qualitative testing.
Usability Measurement Metrics
When testing effectiveness, we ask whether users can complete a given task or achieve their goals using the application. Recommended metrics include, for example:
- Success rate, i.e., the percentage of participants who correctly and completely completed the task
- Catastrophe rate, i.e., the percentage of participants who believe they succeeded but in reality did not complete the task
- Number of errors per unit of time
- Percentage of tasks successfully completed on the first attempt
- Number of requests for help while completing a task
Efficiency tells us how much effort a user needs to complete a task. We can use the following metrics:
- Average time to complete each task (along with range and standard deviation)
- Time required to complete a task on the first attempt
- Time needed to learn a function after it has been changed
- Time required to complete a task compared to an “expert” (a person who knows and controls the application perfectly)
- Time needed to reach “expert” performance
- Number of clicks required to complete a task
- Percentage of time the user relied on a manual
The previous two components of usability can be measured relatively easily. There are many available metrics, and it is often easy to come up with additional ones specific to the application being tested. Satisfaction, however, is more complicated. Satisfaction is the most subjective and hardest-to-measure component of usability. It is essentially the feeling users have after completing a task. Even here, though, there are ways to measure it.
All of them are based on user evaluation. We can use questionnaires or some of the following metrics:
- Ratio of positive to negative words used to describe the application
- Percentage of users who would recommend the application to a friend
- Percentage of users who rated the product as “easier to use” compared to competitors
Among more comprehensive questionnaires, we can mention the System Usability Scale (SUS) or the User Experience Questionnaire (UEQ).
When to Test and What Benefits Does It Bring?
Both types of usability testing are usually included throughout the design and development process, typically at the end of individual design or development cycles, as illustrated in the diagram below. Our ideas or customer requirements are transformed into applications, which are then tested. Data from the testing is used to adjust the initial requirements, which are then implemented and tested again until we are satisfied with the results.
Quantitative testing can take place both during development and after the application has been deployed into production.
Including testing in the design and development process helps resolve usability issues and reduce user frustration from the very beginning. Testing in real-world use then allows us to monitor how usability evolves over time along with new functionality or modifications to existing features. It can help explain why sales have decreased, why some users decided to leave and switch to competing applications, or why process efficiency declined when using internal company systems.
Although usability testing is demanding in terms of time and resources, it can provide application creators with valuable insights into how their product is performing, what can be improved, or why it is not succeeding as it was expected. Even a small amount of testing can have a significant impact on the success of an application.
