Ultimate Guide to Quality Requirements for Software Architects: Meeting Stakeholder Expectations

Table of contents

• A funny story as an introduction to a supposedly boring topic
• An introduction to the ISO standard and a practical look at it
• The alternative: Q42 - the quality model of arc42
• An example non-functional requirement from practise
• Introduction to quality scenarios
• Documenting non-functional requirements
• Further techniques for working with non-functional requirements

Every stakeholder has them: Non-functional requirements or at least (non functional-) expectations for the next software initiative.

• "It has to be fast."
• "It should be easy to maintain."
• "It should be scalable."
• "It should offer a good user experience"
• "A new feature can be developed quickly and easily."

But often no one can explain what they mean by performance efficiency, scalability, usability, coupling, maintainability and so on.

An essential task of a software architect is to ensure that the quality goals of a particular software system become concrete and, hopefully, measurable.

So let's start with an experienced ("funny") story from the real world as an introduction to a supposedly boring quality topic. 👇

A funny story as an introduction to a supposedly boring topic

I am often on the road as a consultant for various CTOs of SMEs. A few years ago, I supported an SME with an initiative for a new software product.

He proudly showed me the list of "non-functional requirements" of this particular software system and said

"Look, we've already identified the future non-functional requirements that we need to fulfil with the new system."

What he showed me was a list of about 30 (!) quality characteristics.

The list looked something like this:

• Scalable
• Reliable
• Usable
• Secure
• Maintainable
• Portable
• Efficient
• Compatible
• Accessible
• Testable
• Flexible
• Interoperable
• Performant
• Resilient
• Modular
• Intuitive
• Customizable
• Responsive
• Consistent
• Fault-tolerant
• Reusable
• Sustainable
• Robust
• Upgradable
• Integrable
• Localizable
• Concise
• High-capacity
• Traceable

This list or these "quality characteristics" were of course completely useless.

Because

1. The entire development team could not even name five quality attributes from this huge list.
2. In the unrealistic case that someone consults the Confluence page with these 30 quality attributes, they would not know what is meant by 'resilient' or these other quality attributes, for example.

I know that this non-functional topic can be boring or annoying because at first glance it doesn't contain any tech stuff.

But as a good software architect - and of course as a good requirements engineer / product owner - you have to do this work and make them concrete and priorisable.

Why?

Because these - and especially these - are your drivers and, more importantly, the reasons for almost all of your software architecture decisions.

But the good news is that in addition to all this boring quality stuff, there are also good and modern techniques for developing and maintaining explicitly non-functional requirements.

An introduction to the ISO standard and a practical look at it

Let's start with the probably boring stuff, the official ISO/IEC 25010 standard.

This quality model is the cornerstone of a system for evaluating system quality. The quality model defines which quality characteristics should be taken into account when evaluating the properties of a software product.

The product quality model defined in ISO/IEC 25010:2023 comprises the eight quality characteristics shown in the following figure:

Here's a brief summary of the ISO/IEC 25010:2023 quality characteristics:

• Functional suitability: Measures the ability of the software to fulfil stated and implied needs.
  • Functional Completeness
  • Functional Correctness
  • Functional Appropriatness
• Performance efficiency: Evaluates performance in relation to the amount of resources consumed.
  • Time Behavior
  • Resource Utilization
  • Capacity
• Compatibility: Evaluates the ability of the software to coexist and exchange information with other products.
  • Co-existence
  • Interoperability
• Interaction capability: Focuses on how easily and satisfactorily users can interact with the software.
  • Appropriateness recognizability
  • Learnability
  • Operability
  • User error protection
  • User engagement
  • User assistence
  • Self-descriptiveness
• Reliability: Examines the ability of software to perform under certain conditions over a certain period of time.
  • Faultlessness
  • Availability
  • Fault Tolerance
  • Recoverability
• Security: Refers to the ability of the software to protect information and data in such a way that persons or systems are only granted access that corresponds to their type and authorization.
  • Confidentiality
  • Integrity
  • Non-repudiation
  • Authenticity
  • Accountability
  • Resistance
• Maintainability: Considers how easily the software can be changed to correct faults, improve performance or adapt to changes in the environment.
  • Modularity
  • Reusability
  • Analysability
  • Modifiability
  • Testability
• Flexibility: The ease of adapting to changes in requirements, context or system environment
  • Adaptability
  • Scalability
  • Installability
  • Replacability
• Safety: Avoid a situation in which human life, health, property or the environment are endangered
  • Operational constraint
  • Risk identification
  • Fail safe
  • Hazard warning
  • Safe integration

Two areas recognizable

The ISO/IEC 25010 standard provides an overview of possible quality characteristics with a useful grouping.

At a high level, we can divide these quality characteristics into two areas:

• the quality of the system at runtime, i.e. the observable behavior. This has a significant influence on the quality of use by users.
• the quality of the system at development time, which has a significant influence on the quality of use for the developers or maintainers of the system.

An opinion

The currently published standard (ISO/IEC 25010:2023) lacks pragmatism and practical applicability, as it omits key terms such as deployability, energy efficiency or code quality and instead focuses on system properties that are described with less intuitive nouns than adjectives. This linguistic choice makes it difficult to discuss and understand quality characteristics in everyday working life.

The alternative: The quality model of arc42

As ISO 25010 lacks practical guidance and pragmatism, there is an alternative approach proposed by arc42 - the arc42 quality model "Q42".

The arc42 Quality Model "Q42" is a straightforward and practical method for assessing product and system quality.

It starts with understanding stakeholder expectations and requirements to identify 8 key system properties. These properties are designed to encompass most of the 100+ traditional quality attributes that are required, desired or expected.

Eight typical stakeholder groups in a software initiative

In this approach, the focus is initially on the stakeholders.

They have identified the typical stakeholder groups of a software initiative, which often have different expectations of the quality of a system:

• Users expect the product to be user-friendly, reliable, readily available and safe.
• Management strives for efficiency in development and operational costs and for reliability.
• Domain experts demand that the system is reliable and efficient in its core functions and can be flexibly expanded with new functionalities.
• Product owners look for flexibility to incorporate new features.
• Developers want a system that is flexible and reliable in terms of code maintenance and is easy to test.
• Testers need a system that is easy to test and delivers consistent results, and value flexibility in automation and instrumentation.
• Administrators value ease of operation and maintenance, integration with other systems, security, flexibility in the execution environment and overall safety.
• Others may have additional expectations that are not explicitly listed here.

Eight typical qualities of a system as #

The quality model proposes eight tags that cover the qualities of a system.

• #reliable: The system should be consistently available, strong, trustworthy, and have the ability to tolerate faults and recover from them, ensuring safety and security.
• #flexible: It should be adaptable during development and in different infrastructure and runtime environments, allowing changes, extensions and integration.
• #efficient: The system is expected to perform quickly and make efficient use of memory, CPU, threading, network, and other resources, and should be maintainable and operable with efficiency.
• #usable: It must be user-friendly, visually appealing and geared to the needs of the user.
• #operable: The system should be easy to install, operate, monitor and maintain, with reasonable operational costs.
• #suitable: It should provide the required features, functionalities, and responsibilities.
• #secure: It should ensure the integrity, confidentiality and accountability of data and protect against unauthorized access.
• #safe: It should be designed to prevent and manage failures, provide timely warning of potential risks and detect them early.

Each of these tags covers a group of related quality attributes. E.g. #reliable covers qualities like

• Accuracy
• Availability
• Backward compatibility
• Capacity
• Clarity
• Compatibility
• Compliance
• Correctness
• Dependability
• Fail safe
• ....

The quality model provides a good scheme to capture relevant qualities of a system by going through these typical eight quality groups and going through the potentially relevant qualities from the perspective of each important stakeholder group.

Based on the identified qualities, you can concretize these qualities with suitable quality scenarios.

An example of a non-functional requirement from practise

Some time ago the PO came to me and said:

"When the user opens the frontend, it must be fast. So it has to be there immediately so that the user can start filling out the form."

If we take a look at Q42, this non-functional requirement can obviously be assigned to the top-level quality property #efficient. If we look at the qualities currently assigned to this top-level group, we find efficiency, which we can probably assign to this "high-level" non-functional requirement.

But let's break down this high-level non-functional requirement that was expected by the product owner.

When I look at the statement from the PO, I notice the following parts:

1. [...] opens the frontend, it must be fast. [...]
2. [...] must not be slow [...] be immediately there [...] that the user can start filling up the form.

Let's create a quality scenario for this example in the next chapter. 👇

Introduction to quality scenarios

Quality scenarios are a great and proven technique to make these lofty and imprecise expectations clear.

Quality scenarios were introduced by Len Bass et al. in the great software architecture book Software Architecture in Practice.

A quality scenario consists of the following elements:

• A stimulus source: a stimulus must have a source. It must come from somewhere. E.g. from a person, a system or another actor
• A stimulus: An event that arrives at a system, e.g. a user action, an attack, a request, ...
• An artifact in a specific environment:
  • The stimulus encounters a specific target - the artifact.
  • The environment is the set of circumstances in which the scenario takes place.
• A response: The response is the activity that takes place as a result of the arrival of the simulus
• A response measure: When the response occurs, it should be measurable in some way so that the scenario can be tested - that is, so that we can determine whether the product team has achieved it.

An example quality scenario for "opening fast the first view of a frontend"

Ok, let's try to write a quality scenario for the PO expectations mentioned.

Defining the stimulus source

The stimulus source in this scenario is the "web app user".

Defining the stimulus

The stimulus or event is "Loading the web app via https://my-web.app".

Defining the artifact and its environment

The artifact is the "web frontend" and the environment is the target browser (e.g. a Chromium based browser)

Defining the response

The response is "The browser loads and renders the web frontend"

Defining the response measure

The response measure is (and this is where explicitness and measurability come in): "The main content of the web front end within the view port is loaded in 2.5 seconds or less."

The resulting quality scenario

The whole resulting quality scenario is pretty cool to discuss with the Product Owner and / or other stakeholders about their expectations.

The quality scenario can be documented in tabular form.

Btw. these quality scenarios can be very easily linked to a developer task or a specific user story.

The great thing about this response measure is that it can be tested automatically via Lighthouse. This metric is called Largest Contentful Paint (LCP).

The LCP and other metrics can be easily measured via Lighthouse.

Tip: If you want to learn more about frontend architecture and quality goals, take a look at the following blog post.

Documenting non-functional requirements

If we look at the arc42 software architecture documentation template, there are two chapters in which the quality goals are mainly documented.

Chapter 1: Introduction and Goals

In this chapter, you document the three (maximum five) top quality goals for the architecture, the fulfillment of which is of utmost importance for the most important stakeholders.
Again, I often refer to the top-level quality groups of Q42 and describe the most demanded quality expectations of the different stakeholders

Chapter 10: Quality Requirements

This is the main chapter in which you document the quality requirements in detail. You can get a good overview with a quality tree, followed by the specifically described quality scenarios.

You can find a good example of a quality tree with a link to the quality scenarios here in the good arc42 example dokchess.

Further techniques for working with non-functional requirements

There are other techniques for working with non-functional requirements in the context of non-functional requirements.

Architecture Characteristics Worksheet

The Architecture Characteristics Worksheet is a tool for identifying and documenting the key characteristics that define the architecture of a system, focusing on aspects such as scalability, performance, reliability and security. It helps stakeholders understand and prioritize architecture requirements by breaking down complex systems into manageable components, each with defined attributes and evaluation metrics. This worksheet is a critical step in the architecture design process, facilitating discussions and decisions that ensure the system achieves its intended goals and performs effectively in its operational environment.

Architecture Inception Canvas

The Architecture Inception Canvas, which is ideal for new software initiatives, has a special section to capture or elaborate the quality goals of a new software initiative.

Architecture Communication Canvas

If you are working out an existing architecture of a system and want to communicate it, you should take a look at the Architecture Communication Canvas. The Architecture Communication Canvas has an extra section for quality requirements, in which you enter the elaborated quality goals on which the software architecture is (probably) based.