During the days when waterfall methodology was in vogue, Non-functional Requirements (NFR) testing was generally the last step before application delivery. For the testers, the entire application offered a complete view of application to test NFRs elicited by the analysts from the business stakeholders. But changes caused by incomplete/inadequately developed NFRs are potential large fixes, requiring costly architectural or design corrections.

Now with adoption of Agile methodologies, the application is built incrementally over multiple sprints. This offers challenges from an NFR implementation and testing perspective. Questions arise related to timing (when and how to implement) and strategy (how best to test in an incremental and combined-regression mode).

Let’s examine how best to deal with Non-functional Requirements in Agile mode.

 

Independent User Story

This means that dedicated technical (user) stories are written to state Non-Functional Requirements explicitly. This would mean that the development and testing is taken care of within the sprint itself. It is likely that regression testing covers this to ensure no impact of changes and to ensure application as a whole is validated.

Examples include:

• As a user, I want the website to be available 99.999 per cent of the times I hit the URL, so that I always use this for making payments
• As a product owner, I want the dashboard displayed to fit large wall mounted display screens, desktops as well as mobile devices
• As a product manager, I need this website in different regional languages to attract customers

Each of these stories expresses a constraint and imparts high visibility which they deserve. But there is a risk that the technical stories sometimes get side-tracked in favour of more attractive user-centric features. If this gets left to the end of the development phase, then this may nullify the benefit from early identifications of the NFR constraint and the advantages that accrue from early development.

 

Acceptance Criteria

NFRs may also be expressed as part of Acceptance criteria which are conditions that a software product must satisfy to be accepted by a user, customer or other stakeholder.

In such cases where the constraint is mentioned as part of acceptance criteria, the effort spent on solutioning NFRs may not be obvious to the stakeholders. For example, a lot of the work required at the framework or architectural is handled by a single acceptance criterion. A user story with some relatively simple feature/s but incorporating a check say for cross-site scripting compliance, may have higher efforts and estimates.

Example:

As a Website User, I want search functionality available on all pages once user is logged in successfully, so that I can search articles using keywords.
Acceptance Criteria:

• Search box should accept alphanumeric values
• Search button should present below search box
• Search results should display only 15 searched items per page
• System responds to all search requests within 10 seconds of receiving the request.(CONSTRAINT)

 

Definition of Done (DoD)

This approach ensures that Non-functional Requirements, implicit or otherwise are handled with the help of explicitly stated and agreed DoD.

Definition of Done (DoD) is a comprehensive checklist of necessary team activities that ensure that only truly done features are delivered, not only in terms of functionality but in terms of quality attributes (i.e. NFR) as well. The advantage of a mention within DoD keeps NFR visible to and therefore actionable by the entire team as and when the increments get developed. It is obvious that the quality attribute will need to be globally applicable to the application for it to be mentioned within the DoD e.g. compatibility, accessibility or usability considerations.

 

Regression Testing

Regression testing is meant for uncovering any unexpected consequences (i.e. new or re-opened bugs) of code changes and modifications through newer user stories. This exercise is also carried out as part of the ‘hardened sprint’ to ensure application is fit for use as a combined unit. It provides a last but sensible opportunity to test NFRs (like application performance related requirements). The phase may represent the ideal time when ethical hacking experts could converge on the scene to perform independent specialized testing alongside the rudimentary functional and non-functional tests.

 

Summary

So the four approaches suggest how the non-functional requirements stay within the team’s purview, get analysed, developed and tested. It must be noted that there is no right or wrong approach that can be specified as a standard. It is to be expected that the agile team can best decide the approach to chosen based on the context existing for the project.

 

Starting a new project or need assistance road-mapping your next steps? Why not take part in our interactive workshop EmbArrk™ and we can help your discover what approach works best for you.